-- Logs begin at Sat 2022-05-28 16:34:50 UTC, end at Sat 2022-05-28 17:50:55 UTC. -- May 28 17:10:36.070511 dv-cinder-843729-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 28 17:10:36.077765 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat May 28 17:10:36 2022] *** May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: nodename: dv-cinder-843729-1-iscsi May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: machine: x86_64 May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: clock source: unix May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: pcre jit disabled May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: detected number of CPU cores: 4 May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: current working directory: / May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: detected binary path: /usr/bin/uwsgi-core May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: your processes number limit is 47680 May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: your memory page size is 4096 bytes May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: detected max file descriptor number: 2048 May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: lock engine: pthread robust mutexes May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: thunder lock: enabled May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 28 17:10:36.083361 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 28 17:10:36.126463 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: Python main interpreter initialized at 0x5613406f79c0 May 28 17:10:36.126463 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: python threads support enabled May 28 17:10:36.126463 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: your server socket listen backlog is limited to 100 connections May 28 17:10:36.126463 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: your mercy for graceful operations on workers is 90 seconds May 28 17:10:36.126814 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: mapped 403077 bytes (393 KB) for 2 cores May 28 17:10:36.126890 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: *** Operational MODE: preforking *** May 28 17:10:36.127175 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: *** uWSGI is running in multiple interpreter mode *** May 28 17:10:36.127175 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: spawned uWSGI master process (pid: 80251) May 28 17:10:36.127400 dv-cinder-843729-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 28 17:10:36.128325 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: spawned uWSGI worker 1 (pid: 80261, cores: 1) May 28 17:10:36.128325 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: spawned uWSGI worker 2 (pid: 80262, cores: 1) May 28 17:10:36.128878 dv-cinder-843729-1-iscsi devstack@keystone.service[80251]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 28 17:10:41.833636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 17:10:41.833636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: from cryptography.utils import int_from_bytes May 28 17:10:41.834407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 17:10:41.834407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: from cryptography.utils import int_from_bytes May 28 17:10:41.881262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 17:10:41.881262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: from cryptography.utils import int_from_bytes May 28 17:10:41.881801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 17:10:41.881801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: from cryptography.utils import int_from_bytes May 28 17:10:41.883147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 17:10:41.883355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 17:10:41.883558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] command line args: [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 17:10:41.883823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 17:10:41.883999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ================================================================================ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 17:10:41.884246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] admin_token = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.884486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] config_dir = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.884669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.884849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] config_source = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.885027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] debug = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.885224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.886335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.886335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.886335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.886335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.886582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.886582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_config_append = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.886767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.886931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_dir = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.887654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_file = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.887894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.888143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.888388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.888635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.888810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.888983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.889223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.889463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.889691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.889919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.890148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.890376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.890616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.890856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.891127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.891634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.891814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] publish_errors = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.892042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.892266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.892441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.892680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.892910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.893140 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] standard_threads = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.893366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.893603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.893776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.893946 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_journal = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.894117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_json = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.894286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_stderr = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.894456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_syslog = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.894681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.894931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.895163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.895443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.895680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.895923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.896155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.896397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.896631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.896871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.897114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.897360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.897589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.897835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.898065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.898305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.898547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.898779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.899025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.899272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.899534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.899767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.900012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.900268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.900516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.900761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.900988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.901215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.901440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.901670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.901913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.902153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.902380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.902606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.903512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.903765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.904002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.904250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.904481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.904714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.904945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.905171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.905399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.905625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.905850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.906083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.906326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.906554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.906788 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.907014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.907245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.907498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.907747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.907974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.908208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.908439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.908670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.908900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.909134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.909358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.909602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.909833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.910066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.910294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.910549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.910782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.911009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.911244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.911497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.911762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.911996 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.912240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.912482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.912714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.912944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.913174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.913417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.913649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.913880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.914117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.914348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.914579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.914809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.915038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.915268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.915545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.915781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.916016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.916246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.916482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.916714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.916944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.917174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.917400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.917629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.917854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.918083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.918311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.918584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.918815 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.919058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.919288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.919566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.919820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.920057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.920283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.external = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.920509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.920798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.921029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.921255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.password = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.921483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.token = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.921717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.921949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.922190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.922416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.922657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.922898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.923126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.923373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.923621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.923865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.924107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.924339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.924575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.924805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.925068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.925317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.925551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.925806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.926084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.926324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.926580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.926818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.927066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.927296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.927556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.927799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.928042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.928268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.928510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.928743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.928990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.929074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 17:10:41.929200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 17:10:41.929276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.929395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] command line args: [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 17:10:41.929473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.929666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 17:10:41.929742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.929853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ================================================================================ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 17:10:41.929937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.930108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] admin_token = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.930259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.930390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_dir = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.930525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.930697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.930840 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.931017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_source = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.931138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.931216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] debug = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.931333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.931450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.931559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.931634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.931634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.931814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.931814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.931991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.932117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.932234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.932317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.932440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.932440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_config_append = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.932632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.932720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.932803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.932876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_dir = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.932992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.933067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.933137 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.933255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.933340 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.933413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.933530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.933615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.933725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.933847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.933934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.934029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.934136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.934253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.934364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.934494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.934604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.934855 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.934943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.935061 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.935136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.935294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.935397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.935614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.935693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.935850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.935928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.936089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.936169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.936326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.936400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.936572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.936801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.936887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.937027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] publish_errors = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.937103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.937259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.937418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.937495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.937565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.937642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.937722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.937884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.937976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.938118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.938204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.938350 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] standard_threads = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.938428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.938582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.938673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.938823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.938900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.939001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.939078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.939177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_journal = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.939267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.939368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_json = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.939538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.939620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_stderr = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.939696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_syslog = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.939767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.939927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 17:10:41.940013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.940181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.940259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.940418 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.940502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.940665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.940752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.940903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.940981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.941145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.941222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.941377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.941463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.941611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.941689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.941842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.941925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.942068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.942154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.942304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.942392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.942549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.942626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.942775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.942860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.943975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.944982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.945145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.945246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.945390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.945518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.945631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.945726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.945872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.945969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.946099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.946189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.946325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.946403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.946551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.946638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.946784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.946869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947754 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.947998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.948085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.948230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.948308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.948480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.948563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.948706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.948796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.948945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.949023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.949175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.949251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950570 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.950896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.951049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.951128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.951283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.951391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.951557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.951642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.951812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.951890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952357 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.952989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.953068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.953225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.953304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.953459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.953546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.953693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.953779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.953927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] role.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] role.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.954961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.955110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.955188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.955378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.955474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.955607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.955694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.955841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.955925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.956069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.956147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.956305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.956392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.956533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.956610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.956778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.956856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.957986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.958139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.958217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.958388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.958466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.958633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.958713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.958876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.958953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.959118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.959195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.959361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.959444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.959606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.959687 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.959840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.959918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.960073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.960154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.960302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.960379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.960534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.960612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.960762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.960848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.961959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.962949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.963100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.963177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.963326 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.963442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.963582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.963664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.963825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.963902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.964066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.964143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.964299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.964385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.964541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.964618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.964773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.964859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.965006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.965084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.965838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.external = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.965929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.966100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.966211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.966346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.966585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.966662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.966822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.966899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.967057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.password = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.967136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.967290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.token = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.967386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.967556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.967644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.967794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.967873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.968972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.969992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.970145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.970225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.970384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.970461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.970613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.970690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.970853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.970933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.971079 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.971157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.971315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.971411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.971570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.971651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.971723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 17:10:41.971864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.972109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.972378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.972621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.972878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.973113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.973349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.973594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.973837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.974067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.974313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.974551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.974809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.975043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.975297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.975556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.975801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.976035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.976281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.976527 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.976760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.976993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.977240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.977479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.977716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.977962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.978213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.978445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.978676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.978907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.979151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.979420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.979670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.979923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.980170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.980440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.980679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.981510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.981918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.982153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.982456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.983106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.983411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.983809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.984141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.984378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.984629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.984884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.985123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.985370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.985620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.985854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.986093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.986325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.986563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.986801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.987042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.987279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.987539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.987789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.988037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.988269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.988501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.988747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.988994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.989229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.989466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.989700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.989931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.990171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.990428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.990658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.990906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.991151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.991422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.991659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.991889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.992123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.992365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.992600 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.992933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.993185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.993436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.993686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.993934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.994182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.994413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.994661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.994904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.995140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.995397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.995639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.995885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.996134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.996371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.997208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.997410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.997629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.997867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.998102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.998339 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.998584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.998825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.999057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.999294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.999559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:41.999809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.000059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.000299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.000548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.000799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.001035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.001281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.001517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.001756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.001990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.002238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.002471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.002721 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.002969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.003214 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.003467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.003555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.003697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.003778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.003917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.003998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.004201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.004436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.004670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.004922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.005167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.005413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.005658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.006062 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.006300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.006551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.006800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.007048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.007296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.007559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.007815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.008067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.008306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.008538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.008769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.009001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.009251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.009506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.009754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.009990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.010226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.010457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.010703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.010940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.011188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.011454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.011711 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.011959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.012242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.012582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.012701 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.013059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.013315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.013556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.013799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.014039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.014289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.014539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.014770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.015020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.015313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.015606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.015852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.016101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.016340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.016581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.016816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.017052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.017287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.017523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.017768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.018002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.018245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.018480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.018727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.018960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.019203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.019461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.019703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.019938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 17:10:42.020108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 17:10:42.040645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.040873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.046318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.046548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.046770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.056888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.076102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.076330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.094513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.114325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.146352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.146580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.152073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.152308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.212725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.212948 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213598 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.213893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.214021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.237821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.237954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.238910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.239039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.239123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.251144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.251292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.263769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.263994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.264933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.265060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.265153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.265276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 17:10:42.283080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.283250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.283364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 17:10:42.283487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 17:10:42.283647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.283817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.283901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 17:10:42.284016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 17:10:42.284098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.284279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.284362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.284537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.284617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.284788 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.284877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.285037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.296139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.296406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.296666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.296940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.297614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.297891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.298158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.298436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.305035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.306644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.308317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.308593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.308871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.309770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.310051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.310309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.310568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.310653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.310858 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.310941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.311138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.311223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.311441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.311527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.311720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.311816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.312089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.312361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.312635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.312908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.313183 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.313451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.338078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.339819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.340095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.340873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.342772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.343052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.345901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.346158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.346434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.348385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.348661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.348940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.353707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.353985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.356522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.356807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.359011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.359431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.359712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.359992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.360261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.360529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.360795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.361058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.361561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.361960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.362242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.362524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.362799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.363070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.363351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.363621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.380126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.380394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.382738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.383011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.385075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.388428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.389597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.389887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.392110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.392405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.394914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.397434 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.398531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.401095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.402331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.402640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.403055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.403366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.403641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.403918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.404976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.405273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.405561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.405844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.406129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.406414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.422847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.423122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.423402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.423662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.425494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.425776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.426040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.426300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.431782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.433483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.433752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.434892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.436624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.436899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.437521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.437796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.438068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.438339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.438609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.439991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.440272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.440553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.440832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.441113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.452691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.452965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.453241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.453521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.453795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.454062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.454340 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.455228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.455527 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.455808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.456094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.456372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.456639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.456915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.472472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.475100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.475737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.478392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.479025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.480648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.482369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.483089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.483394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.483673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.484261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.485878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.486175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.486457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.491083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.492655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.493926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.495574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.495900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.496184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.496578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.496903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.498867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.499154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.499465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.499752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.505963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.506242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.506617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.508924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.509206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.509584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.515052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.515323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.515618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.515885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.516147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.516412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.516694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.516955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.517232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.517499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.517762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.518015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.518105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 17:10:42.518270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.518352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.518520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.518656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.518857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.519124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.519430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.519705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.519982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.520249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.520522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.520789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.521048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.521316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.521592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 17:10:42.557990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5613406f79c0 pid: 80262 (default app) May 28 17:10:42.561084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5613406f79c0 pid: 80261 (default app) May 28 17:10:42.561437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f431390-e46b-4bbc-9a2b-86bde8f1fdbc None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:42.561731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f431390-e46b-4bbc-9a2b-86bde8f1fdbc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:42.562006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f431390-e46b-4bbc-9a2b-86bde8f1fdbc None None] PATH_INFO: `/v3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:42.566257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Sat May 28 17:10:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:10:50.501211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6357c-1a7e-4fa7-85e8-9a4b238c2a28 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:50.501780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6357c-1a7e-4fa7-85e8-9a4b238c2a28 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:50.501780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6357c-1a7e-4fa7-85e8-9a4b238c2a28 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:50.506196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:10:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:10:50.511484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:50.512530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:50.512676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:50.579960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: INFO dbcounter [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] Registered counter for database keystone May 28 17:10:50.600915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG oslo_db.sqlalchemy.engines [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80262) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 17:10:50.607280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writer thread running {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 17:10:50.796126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG passlib.handlers.bcrypt [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80262) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 28 17:10:50.796473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG passlib.handlers.bcrypt [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80262) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 28 17:10:50.851164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:10:50.852788 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:10:50.978148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:10:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 28 17:10:51.164594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6357c-1a7e-4fa7-85e8-9a4b238c2a28 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:51.233366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: INFO dbcounter [None req-26e6357c-1a7e-4fa7-85e8-9a4b238c2a28 None None] Registered counter for database keystone May 28 17:10:51.252431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG oslo_db.sqlalchemy.engines [None req-26e6357c-1a7e-4fa7-85e8-9a4b238c2a28 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80261) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 17:10:51.258632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writer thread running {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 17:10:51.385519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6357c-1a7e-4fa7-85e8-9a4b238c2a28 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:51.389845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dacec-4b97-4e4a-af30-b84582a27398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:51.390519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dacec-4b97-4e4a-af30-b84582a27398 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:51.390805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dacec-4b97-4e4a-af30-b84582a27398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:51.391081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dacec-4b97-4e4a-af30-b84582a27398 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:51.399210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7dacec-4b97-4e4a-af30-b84582a27398 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:51.450971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2b7dacec-4b97-4e4a-af30-b84582a27398 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 28 17:10:51.451311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7dacec-4b97-4e4a-af30-b84582a27398 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:51.460838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Sat May 28 17:10:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:10:51.467750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:51.482652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d00ab0-9083-41b2-9ca9-320e34b8df41 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:51.487300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fdb81-27c4-43e1-977e-ce20af98cc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:51.488445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fdb81-27c4-43e1-977e-ce20af98cc77 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:51.488731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fdb81-27c4-43e1-977e-ce20af98cc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:51.489007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fdb81-27c4-43e1-977e-ce20af98cc77 admin admin] PATH_INFO: `/v3/projects/3b7d62044f5549a6b53772308a090b05` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:51.490560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fdb81-27c4-43e1-977e-ce20af98cc77 admin admin] RBAC: Authorizing `identity:get_project(project_id=3b7d62044f5549a6b53772308a090b05)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:51.648478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING keystone.common.rbac_enforcer.enforcer [None req-406fdb81-27c4-43e1-977e-ce20af98cc77 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 28 17:10:51.649126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fdb81-27c4-43e1-977e-ce20af98cc77 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:51.653637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Sat May 28 17:10:51 2022] GET /identity/v3/projects/3b7d62044f5549a6b53772308a090b05 => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:10:53.826924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99304296-3235-458d-a4a3-17fd8b50aa57 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:53.827452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99304296-3235-458d-a4a3-17fd8b50aa57 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:53.827559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99304296-3235-458d-a4a3-17fd8b50aa57 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:53.831598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:10:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:10:53.837000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fcb4d-d699-4a3f-b2c0-11968529a4c0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:53.837413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fcb4d-d699-4a3f-b2c0-11968529a4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:53.837553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fcb4d-d699-4a3f-b2c0-11968529a4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:53.856783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b19fcb4d-d699-4a3f-b2c0-11968529a4c0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:10:53.858527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b19fcb4d-d699-4a3f-b2c0-11968529a4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:10:53.866210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:10:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 28 17:10:53.889559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99304296-3235-458d-a4a3-17fd8b50aa57 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:53.901671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99304296-3235-458d-a4a3-17fd8b50aa57 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:53.906654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c693b-d0f3-4257-a01f-594cca75af7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:53.907289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c693b-d0f3-4257-a01f-594cca75af7f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:53.907610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c693b-d0f3-4257-a01f-594cca75af7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:53.907895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c693b-d0f3-4257-a01f-594cca75af7f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:53.915645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c693b-d0f3-4257-a01f-594cca75af7f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:53.917431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c693b-d0f3-4257-a01f-594cca75af7f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:53.926135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Sat May 28 17:10:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:10:53.931253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fcb4d-d699-4a3f-b2c0-11968529a4c0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:53.940343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fcb4d-d699-4a3f-b2c0-11968529a4c0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:53.944609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da75fc-747d-4467-8136-486e5462f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:53.945457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da75fc-747d-4467-8136-486e5462f41e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:53.945735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da75fc-747d-4467-8136-486e5462f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:53.946007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da75fc-747d-4467-8136-486e5462f41e admin admin] PATH_INFO: `/v3/users/04bc439d29964fc096fdffcc884a8f66` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:53.947422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da75fc-747d-4467-8136-486e5462f41e admin admin] RBAC: Authorizing `identity:get_user(user_id=04bc439d29964fc096fdffcc884a8f66)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:53.949744 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da75fc-747d-4467-8136-486e5462f41e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:53.951489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Sat May 28 17:10:53 2022] GET /identity/v3/users/04bc439d29964fc096fdffcc884a8f66 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:10:56.225465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffe024-a9c6-4734-9b1b-c9409c1972cf None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.225993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffe024-a9c6-4734-9b1b-c9409c1972cf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.226098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffe024-a9c6-4734-9b1b-c9409c1972cf None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.227414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:10:56.233415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3ec77-7ecd-4d51-8ff2-b2fa0d562207 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.233826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3ec77-7ecd-4d51-8ff2-b2fa0d562207 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.234109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3ec77-7ecd-4d51-8ff2-b2fa0d562207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.251103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5cc3ec77-7ecd-4d51-8ff2-b2fa0d562207 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:10:56.252317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5cc3ec77-7ecd-4d51-8ff2-b2fa0d562207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:10:56.285357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820dc0e-30f0-43a6-a515-687e18c417fa None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.285648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820dc0e-30f0-43a6-a515-687e18c417fa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.285918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820dc0e-30f0-43a6-a515-687e18c417fa None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.287247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:10:56.292073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.292357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.292634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.331786 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:10:56.332565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG passlib.handlers.bcrypt [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80261) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 28 17:10:56.333082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG passlib.handlers.bcrypt [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80261) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 28 17:10:56.354021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3ec77-7ecd-4d51-8ff2-b2fa0d562207 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:56.377567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:10:56.380090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:10:56.381710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:10:56.461929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3ec77-7ecd-4d51-8ff2-b2fa0d562207 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:56.487662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:10:56.497494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f5764-05db-4bec-bd33-62b07c83f2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:56.498298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5764-05db-4bec-bd33-62b07c83f2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.498585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5764-05db-4bec-bd33-62b07c83f2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.498865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5764-05db-4bec-bd33-62b07c83f2e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.509245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:56.509939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f5764-05db-4bec-bd33-62b07c83f2e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-826f5764-05db-4bec-bd33-62b07c83f2e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:10:56.521392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:10:56.522582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:10:56.523466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:10:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:10:56.529050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f5764-05db-4bec-bd33-62b07c83f2e3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:56.610739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:10:56.611027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5287-c8e6-40d3-933e-db12b90ec34d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:56.613335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:10:56.631391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f5764-05db-4bec-bd33-62b07c83f2e3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:56.647950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c07d2-e9cd-4e3a-9176-4a8a5a7b00be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:56.648853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c07d2-e9cd-4e3a-9176-4a8a5a7b00be None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.649142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c07d2-e9cd-4e3a-9176-4a8a5a7b00be None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.649422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c07d2-e9cd-4e3a-9176-4a8a5a7b00be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.665265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:56.665930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.666215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.666491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.667245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:10:56.667587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:56.669719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:56.681558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:10:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:10:56.689263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:56.722735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c07d2-e9cd-4e3a-9176-4a8a5a7b00be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:56.724302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c07d2-e9cd-4e3a-9176-4a8a5a7b00be None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:56.795797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d0330-1e6c-4d6e-8ea0-80d109bf3742 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:56.821282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 17:10:56.827710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c07d2-e9cd-4e3a-9176-4a8a5a7b00be None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:56.829174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f99989-2791-4f05-afbc-eeb6fcbea487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:56.830038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f99989-2791-4f05-afbc-eeb6fcbea487 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.830320 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f99989-2791-4f05-afbc-eeb6fcbea487 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.830486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:10:56.830594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f99989-2791-4f05-afbc-eeb6fcbea487 None admin] PATH_INFO: `/v3/users/04bc439d29964fc096fdffcc884a8f66` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.831835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f99989-2791-4f05-afbc-eeb6fcbea487 None admin] RBAC: Authorizing `identity:get_user(user_id=04bc439d29964fc096fdffcc884a8f66)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:56.833283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f99989-2791-4f05-afbc-eeb6fcbea487 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:56.834905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:10:56 2022] GET /identity/v3/users/04bc439d29964fc096fdffcc884a8f66 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:10:56.840366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f99989-2791-4f05-afbc-eeb6fcbea487 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:56.855097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:10:56.925179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c07d2-e9cd-4e3a-9176-4a8a5a7b00be None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:56.939748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f99989-2791-4f05-afbc-eeb6fcbea487 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:56.960343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:56.961494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.961787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.962069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.974363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9c26d-db70-43a9-9ecd-9173c8cddb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:56.975161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9c26d-db70-43a9-9ecd-9173c8cddb18 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:56.975470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9c26d-db70-43a9-9ecd-9173c8cddb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:56.975747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9c26d-db70-43a9-9ecd-9173c8cddb18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:56.976929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9c26d-db70-43a9-9ecd-9173c8cddb18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:56.978386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9c26d-db70-43a9-9ecd-9173c8cddb18 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:56.980035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 10/18] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:10:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:10:56.982372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:56.984140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:56.986811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9c26d-db70-43a9-9ecd-9173c8cddb18 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:10:57.006763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:10:57.007723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:10:57.008528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:10:57.008528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:10:57.008528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:10:57.008528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:10:57.008528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 9/19] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:10:57.014143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:57.067803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:10:57.073606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:10:57.087722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9c26d-db70-43a9-9ecd-9173c8cddb18 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:57.118074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd0f7-6b2c-4ce7-b19a-6c31109c010a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:57.122739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653189a-adde-4e72-8401-f7b8a2d81e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:57.123508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653189a-adde-4e72-8401-f7b8a2d81e90 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:57.123793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653189a-adde-4e72-8401-f7b8a2d81e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:57.124070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653189a-adde-4e72-8401-f7b8a2d81e90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:57.124910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653189a-adde-4e72-8401-f7b8a2d81e90 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.domain.id=default, user.id=04bc439d29964fc096fdffcc884a8f66) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:10:57.125222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653189a-adde-4e72-8401-f7b8a2d81e90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:57.126726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653189a-adde-4e72-8401-f7b8a2d81e90 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:57.134391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Sat May 28 17:10:56 2022] GET /identity/v3/role_assignments?user.id=04bc439d29964fc096fdffcc884a8f66&scope.domain.id=default&role.id=e07cc5009f624eb9912a837c934e68aa => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:10:57.264608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076a157-8e10-4f69-8987-97596dd5de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:57.265299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a157-8e10-4f69-8987-97596dd5de5c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:57.265576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a157-8e10-4f69-8987-97596dd5de5c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:57.265852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a157-8e10-4f69-8987-97596dd5de5c None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:57.266606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076a157-8e10-4f69-8987-97596dd5de5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:10:57.266920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076a157-8e10-4f69-8987-97596dd5de5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:10:57.268389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076a157-8e10-4f69-8987-97596dd5de5c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:10:57.287751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:10:59.487133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8c81f-f9e0-4b3a-85ae-6a7e307f8df9 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:59.487706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8c81f-f9e0-4b3a-85ae-6a7e307f8df9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:59.487706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8c81f-f9e0-4b3a-85ae-6a7e307f8df9 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:59.489071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:10:59.494496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b0bf-31e3-4f62-930c-c613e2cdbac5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:59.494908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b0bf-31e3-4f62-930c-c613e2cdbac5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:59.495093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b0bf-31e3-4f62-930c-c613e2cdbac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:59.524351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-66b7b0bf-31e3-4f62-930c-c613e2cdbac5 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:10:59.525550 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-66b7b0bf-31e3-4f62-930c-c613e2cdbac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:10:59.613880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:10:59.646514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8c81f-f9e0-4b3a-85ae-6a7e307f8df9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:59.807296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8c81f-f9e0-4b3a-85ae-6a7e307f8df9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:10:59.850866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fecae-8244-4078-b1a2-df0e556a17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:10:59.851709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fecae-8244-4078-b1a2-df0e556a17a3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:10:59.852002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fecae-8244-4078-b1a2-df0e556a17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:10:59.852279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fecae-8244-4078-b1a2-df0e556a17a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:10:59.865543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fecae-8244-4078-b1a2-df0e556a17a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c13fecae-8244-4078-b1a2-df0e556a17a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:10:59.877567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:10:59.878463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:10:59.879223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 13/24] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:10:59.888724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7b0bf-31e3-4f62-930c-c613e2cdbac5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:10:59.960361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:00.038895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7b0bf-31e3-4f62-930c-c613e2cdbac5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:00.091781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70365514-ac65-4b56-8c4f-ed74de0c13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:00.092390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70365514-ac65-4b56-8c4f-ed74de0c13e0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.092671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70365514-ac65-4b56-8c4f-ed74de0c13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.092949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70365514-ac65-4b56-8c4f-ed74de0c13e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.093710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70365514-ac65-4b56-8c4f-ed74de0c13e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:00.094021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70365514-ac65-4b56-8c4f-ed74de0c13e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:00.095438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70365514-ac65-4b56-8c4f-ed74de0c13e0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:00.112221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 12/25] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:10:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:00.117889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fecae-8244-4078-b1a2-df0e556a17a3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:00.129377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:00.239309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9e37-29dc-451c-b805-701a258652c8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.239621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9e37-29dc-451c-b805-701a258652c8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.240118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9e37-29dc-451c-b805-701a258652c8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.241639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 13/26] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:11:00.252776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6886-13e8-4f2e-b894-f998e1be9507 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.253314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6886-13e8-4f2e-b894-f998e1be9507 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.253782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6886-13e8-4f2e-b894-f998e1be9507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.262884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fecae-8244-4078-b1a2-df0e556a17a3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:00.298394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c2ec6886-13e8-4f2e-b894-f998e1be9507 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:00.299689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c2ec6886-13e8-4f2e-b894-f998e1be9507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:00.305903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabb459-9563-45fe-9a97-93dceecc50a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:00.306814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabb459-9563-45fe-9a97-93dceecc50a0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.307093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabb459-9563-45fe-9a97-93dceecc50a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.307390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabb459-9563-45fe-9a97-93dceecc50a0 None admin] PATH_INFO: `/v3/users/04bc439d29964fc096fdffcc884a8f66` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.314656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabb459-9563-45fe-9a97-93dceecc50a0 None admin] RBAC: Authorizing `identity:get_user(user_id=04bc439d29964fc096fdffcc884a8f66)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:00.316205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabb459-9563-45fe-9a97-93dceecc50a0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:00.317934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 14/27] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:00 2022] GET /identity/v3/users/04bc439d29964fc096fdffcc884a8f66 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:00.325673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabb459-9563-45fe-9a97-93dceecc50a0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:00.449939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 14/28] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:00.469615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:00.484027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec6886-13e8-4f2e-b894-f998e1be9507 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:00.535130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabb459-9563-45fe-9a97-93dceecc50a0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:00.578687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:00.614218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c8bd4-51a7-4a1b-87b4-59d313a2ee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:00.615311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c8bd4-51a7-4a1b-87b4-59d313a2ee81 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.615806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c8bd4-51a7-4a1b-87b4-59d313a2ee81 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.616247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c8bd4-51a7-4a1b-87b4-59d313a2ee81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.618592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c8bd4-51a7-4a1b-87b4-59d313a2ee81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:00.620240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c8bd4-51a7-4a1b-87b4-59d313a2ee81 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:00.624243 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 15/29] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:00.629829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c8bd4-51a7-4a1b-87b4-59d313a2ee81 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:00.682548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec6886-13e8-4f2e-b894-f998e1be9507 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:00.724852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d47d-44e0-4035-b89e-bd6d1665d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:00.725987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d47d-44e0-4035-b89e-bd6d1665d4fe None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.726435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d47d-44e0-4035-b89e-bd6d1665d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.726871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d47d-44e0-4035-b89e-bd6d1665d4fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.739244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d47d-44e0-4035-b89e-bd6d1665d4fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:00.741063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d47d-44e0-4035-b89e-bd6d1665d4fe None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:00.751483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c8bd4-51a7-4a1b-87b4-59d313a2ee81 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bde9d47d-44e0-4035-b89e-bd6d1665d4fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:11:00.760457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:00.761264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:00.762594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:11:00.762594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:11:00.762594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:11:00.762594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:00.765074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:00.766751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aaee5-2ff4-404e-b617-c189a4a58763 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.767246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aaee5-2ff4-404e-b617-c189a4a58763 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.767747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aaee5-2ff4-404e-b617-c189a4a58763 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.769362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 16/31] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:00.770725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258abd-d84c-498d-99d6-6db0c1b863e8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.771226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258abd-d84c-498d-99d6-6db0c1b863e8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.771719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258abd-d84c-498d-99d6-6db0c1b863e8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.773241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 17/32] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:00.774311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258abd-d84c-498d-99d6-6db0c1b863e8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:00.792955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2322a58-e2e9-42b8-9bea-018f45719b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:00.793787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322a58-e2e9-42b8-9bea-018f45719b22 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.794074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322a58-e2e9-42b8-9bea-018f45719b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.794358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322a58-e2e9-42b8-9bea-018f45719b22 None admin] PATH_INFO: `/v3/domains/default/users/04bc439d29964fc096fdffcc884a8f66/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.796600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2322a58-e2e9-42b8-9bea-018f45719b22 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=04bc439d29964fc096fdffcc884a8f66, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:00.798246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2322a58-e2e9-42b8-9bea-018f45719b22 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:00.813951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:00.894193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:00.906808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258abd-d84c-498d-99d6-6db0c1b863e8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:00.943190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 16/33] 10.222.0.33 () {58 vars in 1361 bytes} [Sat May 28 17:11:00 2022] PUT /identity/v3/domains/default/users/04bc439d29964fc096fdffcc884a8f66/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:00.945337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3748-7c94-4220-867f-b980305d3ed5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.945866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3748-7c94-4220-867f-b980305d3ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.946383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3748-7c94-4220-867f-b980305d3ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.948171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40f093-e40b-417c-bf25-6e8f58f9947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:00.948852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f093-e40b-417c-bf25-6e8f58f9947e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.949305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f093-e40b-417c-bf25-6e8f58f9947e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.949627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f093-e40b-417c-bf25-6e8f58f9947e None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.950575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40f093-e40b-417c-bf25-6e8f58f9947e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:00.950900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40f093-e40b-417c-bf25-6e8f58f9947e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:00.952380 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40f093-e40b-417c-bf25-6e8f58f9947e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:00.984687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 18/34] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:00.988673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ce1ca-4da6-4056-a769-d693e87a8ace None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:00.990145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-15fc3748-7c94-4220-867f-b980305d3ed5 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:00.990657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ce1ca-4da6-4056-a769-d693e87a8ace None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:00.991161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ce1ca-4da6-4056-a769-d693e87a8ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:00.994828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-15fc3748-7c94-4220-867f-b980305d3ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:01.026788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-760ce1ca-4da6-4056-a769-d693e87a8ace None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:01.032593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-760ce1ca-4da6-4056-a769-d693e87a8ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:01.101622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 17/35] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:01.102369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc3748-7c94-4220-867f-b980305d3ed5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:01.123967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 19/36] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:01.125759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58611fa-6427-4ac0-a467-e5acf3cffc07 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:01.126258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58611fa-6427-4ac0-a467-e5acf3cffc07 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:01.126846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58611fa-6427-4ac0-a467-e5acf3cffc07 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:01.128392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 20/37] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:01.135489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58611fa-6427-4ac0-a467-e5acf3cffc07 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:01.228301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc3748-7c94-4220-867f-b980305d3ed5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:01.237009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:01.258673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58611fa-6427-4ac0-a467-e5acf3cffc07 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:01.271030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:01.280616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97e660-e036-4613-97df-fd7ab99a9aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:01.281302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97e660-e036-4613-97df-fd7ab99a9aae None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:01.281592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97e660-e036-4613-97df-fd7ab99a9aae None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:01.281879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97e660-e036-4613-97df-fd7ab99a9aae None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:01.283299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b97e660-e036-4613-97df-fd7ab99a9aae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:01.285634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b97e660-e036-4613-97df-fd7ab99a9aae None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:01.298017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346770-1b39-4902-b911-902648bffd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:01.298783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346770-1b39-4902-b911-902648bffd05 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:01.299134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346770-1b39-4902-b911-902648bffd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:01.299528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346770-1b39-4902-b911-902648bffd05 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:01.300360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346770-1b39-4902-b911-902648bffd05 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:01.301814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346770-1b39-4902-b911-902648bffd05 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:01.327329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 18/38] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:11:01 2022] POST /identity/v3/projects => generated 301 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:01.328861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e0ada-29ce-4b33-92c2-d52009248b8e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:01.329146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e0ada-29ce-4b33-92c2-d52009248b8e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:01.329429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e0ada-29ce-4b33-92c2-d52009248b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:01.350296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dd1e0ada-29ce-4b33-92c2-d52009248b8e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:01.351644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dd1e0ada-29ce-4b33-92c2-d52009248b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:01.436743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 19/39] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:01.437692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e0ada-29ce-4b33-92c2-d52009248b8e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:01.442346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 21/40] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 17:11:01 2022] POST /identity/v3/roles => generated 225 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:01.481457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346770-1b39-4902-b911-902648bffd05 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:01.572030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e0ada-29ce-4b33-92c2-d52009248b8e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:01.591926 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:01.623831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb810a-3e57-46c8-bd99-104335026086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:01.624633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb810a-3e57-46c8-bd99-104335026086 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:01.624925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb810a-3e57-46c8-bd99-104335026086 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:01.625210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb810a-3e57-46c8-bd99-104335026086 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:01.626064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb810a-3e57-46c8-bd99-104335026086 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:01.627546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb810a-3e57-46c8-bd99-104335026086 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:01.658834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 20/41] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 17:11:01 2022] POST /identity/v3/roles => generated 219 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:01.744011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346770-1b39-4902-b911-902648bffd05 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:01.789228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13b5cf-ddba-4281-b0a3-235350bac800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:01.789940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b5cf-ddba-4281-b0a3-235350bac800 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:01.790284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b5cf-ddba-4281-b0a3-235350bac800 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:01.790630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b5cf-ddba-4281-b0a3-235350bac800 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:01.791517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13b5cf-ddba-4281-b0a3-235350bac800 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:01.793287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13b5cf-ddba-4281-b0a3-235350bac800 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:01.805826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:01.812879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 22/42] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 17:11:01 2022] POST /identity/v3/roles => generated 223 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:11:01.971794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940239c5-9994-4b0d-8bb9-9535c3f84838 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:01.972084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940239c5-9994-4b0d-8bb9-9535c3f84838 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:01.972373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940239c5-9994-4b0d-8bb9-9535c3f84838 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:01.973776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 21/43] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:01.979796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d847-75cc-4542-8572-95b45ff9c1a8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:01.980098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d847-75cc-4542-8572-95b45ff9c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:01.980470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d847-75cc-4542-8572-95b45ff9c1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:02.005607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-de94d847-75cc-4542-8572-95b45ff9c1a8 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:02.007231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de94d847-75cc-4542-8572-95b45ff9c1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:02.084119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 23/44] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:02.108356 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940239c5-9994-4b0d-8bb9-9535c3f84838 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:02.208778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940239c5-9994-4b0d-8bb9-9535c3f84838 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:02.241881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79118bca-71d4-4a98-89e4-e8df8cdc5280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:02.242604 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118bca-71d4-4a98-89e4-e8df8cdc5280 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:02.242883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118bca-71d4-4a98-89e4-e8df8cdc5280 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:02.243160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118bca-71d4-4a98-89e4-e8df8cdc5280 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:02.244351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79118bca-71d4-4a98-89e4-e8df8cdc5280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:02.245862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79118bca-71d4-4a98-89e4-e8df8cdc5280 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:02.247794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 22/45] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:02.253221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94d847-75cc-4542-8572-95b45ff9c1a8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:02.351102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94d847-75cc-4542-8572-95b45ff9c1a8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:02.383131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:02.389809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7d78e-74b6-413c-aadb-9cf866f9c877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:02.390490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d78e-74b6-413c-aadb-9cf866f9c877 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:02.390778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d78e-74b6-413c-aadb-9cf866f9c877 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:02.391060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d78e-74b6-413c-aadb-9cf866f9c877 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:02.392077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7d78e-74b6-413c-aadb-9cf866f9c877 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:02.393566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7d78e-74b6-413c-aadb-9cf866f9c877 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:02.421396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 24/46] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:11:02 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:03.421755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464f44-af99-4227-9afa-37158c21e2ef None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:03.422270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464f44-af99-4227-9afa-37158c21e2ef None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:03.422351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464f44-af99-4227-9afa-37158c21e2ef None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:03.423709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 23/47] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:03.428913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905a4d4-1eb1-4320-91f7-1206c6988eb0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:03.429307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905a4d4-1eb1-4320-91f7-1206c6988eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:03.429645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905a4d4-1eb1-4320-91f7-1206c6988eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:03.446271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c905a4d4-1eb1-4320-91f7-1206c6988eb0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:03.447495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c905a4d4-1eb1-4320-91f7-1206c6988eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:03.513323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 25/48] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:03.536573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74464f44-af99-4227-9afa-37158c21e2ef None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:03.580225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:03.639713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74464f44-af99-4227-9afa-37158c21e2ef None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:03.676141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05a613-a62a-4eb7-a66b-bc6f632df26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:03.676862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05a613-a62a-4eb7-a66b-bc6f632df26f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:03.677146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05a613-a62a-4eb7-a66b-bc6f632df26f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:03.677424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05a613-a62a-4eb7-a66b-bc6f632df26f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:03.688255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05a613-a62a-4eb7-a66b-bc6f632df26f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ea05a613-a62a-4eb7-a66b-bc6f632df26f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:03.695832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:03.696825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:03.698054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 24/49] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:03.702760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905a4d4-1eb1-4320-91f7-1206c6988eb0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:03.806265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905a4d4-1eb1-4320-91f7-1206c6988eb0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:03.819585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:03.842323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:03.842999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:03.843288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:03.843607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:03.844365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:03.844677 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:03.846063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:03.854123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 26/50] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:03.860468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05a613-a62a-4eb7-a66b-bc6f632df26f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:03.966224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:03.969964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05a613-a62a-4eb7-a66b-bc6f632df26f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:04.009247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b866c-d6c1-4df6-b2e5-a9ca8b645a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:04.010121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b866c-d6c1-4df6-b2e5-a9ca8b645a12 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:04.010411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b866c-d6c1-4df6-b2e5-a9ca8b645a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:04.010691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b866c-d6c1-4df6-b2e5-a9ca8b645a12 None admin] PATH_INFO: `/v3/users/04bc439d29964fc096fdffcc884a8f66` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:04.012003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b866c-d6c1-4df6-b2e5-a9ca8b645a12 None admin] RBAC: Authorizing `identity:get_user(user_id=04bc439d29964fc096fdffcc884a8f66)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:04.013484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b866c-d6c1-4df6-b2e5-a9ca8b645a12 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:04.015120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 25/51] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:03 2022] GET /identity/v3/users/04bc439d29964fc096fdffcc884a8f66 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:04.020527 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:04.121133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56248-711a-40c4-8a85-5ec89e99388b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:04.154678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c79cc-5552-419b-a1bb-968513532fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:04.155585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c79cc-5552-419b-a1bb-968513532fdf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:04.155874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c79cc-5552-419b-a1bb-968513532fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:04.156153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c79cc-5552-419b-a1bb-968513532fdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:04.157334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c79cc-5552-419b-a1bb-968513532fdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:04.158791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c79cc-5552-419b-a1bb-968513532fdf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:04.160461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 27/52] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:04.166054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b866c-d6c1-4df6-b2e5-a9ca8b645a12 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:04.265176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b866c-d6c1-4df6-b2e5-a9ca8b645a12 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:04.298375 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d95984-2065-4857-b966-17c07f5e579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:04.299034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d95984-2065-4857-b966-17c07f5e579f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:04.299323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d95984-2065-4857-b966-17c07f5e579f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:04.299626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d95984-2065-4857-b966-17c07f5e579f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:04.300467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d95984-2065-4857-b966-17c07f5e579f None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.domain.id=default, user.id=04bc439d29964fc096fdffcc884a8f66) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:04.300791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d95984-2065-4857-b966-17c07f5e579f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:04.302251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d95984-2065-4857-b966-17c07f5e579f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:04.308216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 26/53] 10.222.0.33 () {56 vars in 1322 bytes} [Sat May 28 17:11:04 2022] GET /identity/v3/role_assignments?user.id=04bc439d29964fc096fdffcc884a8f66&scope.domain.id=default&role.id=e07cc5009f624eb9912a837c934e68aa => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:04.652681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075ebf-a102-4502-802c-09c3b6b109aa None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:04.653173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075ebf-a102-4502-802c-09c3b6b109aa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:04.653270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075ebf-a102-4502-802c-09c3b6b109aa None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:04.654635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 28/54] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:04.659875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1958f7-2220-49d0-ac21-65a520db241d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:04.660235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1958f7-2220-49d0-ac21-65a520db241d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:04.660436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1958f7-2220-49d0-ac21-65a520db241d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:04.677708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7c1958f7-2220-49d0-ac21-65a520db241d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:04.679155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7c1958f7-2220-49d0-ac21-65a520db241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:04.704582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:04.746356 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 27/55] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:04.768583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075ebf-a102-4502-802c-09c3b6b109aa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:04.812497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:04.868685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075ebf-a102-4502-802c-09c3b6b109aa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:04.900610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c58c7b-020f-4926-b7a7-62371455b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:04.901398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c58c7b-020f-4926-b7a7-62371455b399 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:04.901681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c58c7b-020f-4926-b7a7-62371455b399 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:04.902469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c58c7b-020f-4926-b7a7-62371455b399 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:04.903903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c58c7b-020f-4926-b7a7-62371455b399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:04.905311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c58c7b-020f-4926-b7a7-62371455b399 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:04.906906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 29/56] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:04.912432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1958f7-2220-49d0-ac21-65a520db241d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:05.007831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1958f7-2220-49d0-ac21-65a520db241d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:05.040793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251edc03-83d4-47f9-b9d5-d8c51b1b582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:05.041443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251edc03-83d4-47f9-b9d5-d8c51b1b582c None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:05.041730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251edc03-83d4-47f9-b9d5-d8c51b1b582c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:05.042011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251edc03-83d4-47f9-b9d5-d8c51b1b582c None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:05.043367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251edc03-83d4-47f9-b9d5-d8c51b1b582c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:05.044824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251edc03-83d4-47f9-b9d5-d8c51b1b582c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:05.068994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 28/57] 10.222.0.33 () {60 vars in 1140 bytes} [Sat May 28 17:11:04 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:07.251714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c3cbb-20be-4d2f-82af-810be1f5c699 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:07.252229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c3cbb-20be-4d2f-82af-810be1f5c699 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:07.252313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c3cbb-20be-4d2f-82af-810be1f5c699 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:07.253640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 30/58] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:07.258883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c69f-9257-4da2-86c7-c84af869f237 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:07.259285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c69f-9257-4da2-86c7-c84af869f237 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:07.259478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c69f-9257-4da2-86c7-c84af869f237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:07.275897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dff9c69f-9257-4da2-86c7-c84af869f237 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:07.277110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dff9c69f-9257-4da2-86c7-c84af869f237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:07.286023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:07.343234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 29/59] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:07.367786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c3cbb-20be-4d2f-82af-810be1f5c699 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:07.467654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c3cbb-20be-4d2f-82af-810be1f5c699 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:07.477903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:07.501842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513f9ac-0060-450d-ae94-7a9792c16b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:07.502569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f9ac-0060-450d-ae94-7a9792c16b3e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:07.502852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f9ac-0060-450d-ae94-7a9792c16b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:07.503124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f9ac-0060-450d-ae94-7a9792c16b3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:07.504329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513f9ac-0060-450d-ae94-7a9792c16b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:07.505769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513f9ac-0060-450d-ae94-7a9792c16b3e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:07.507395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 31/60] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:07.513014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9c69f-9257-4da2-86c7-c84af869f237 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:07.609600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9c69f-9257-4da2-86c7-c84af869f237 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:07.642334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe74b80-8a90-418f-aeb3-31a3552c4510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:07.642997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74b80-8a90-418f-aeb3-31a3552c4510 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:07.643283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74b80-8a90-418f-aeb3-31a3552c4510 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:07.643592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74b80-8a90-418f-aeb3-31a3552c4510 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:07.644309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe74b80-8a90-418f-aeb3-31a3552c4510 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:07.645764 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe74b80-8a90-418f-aeb3-31a3552c4510 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:07.666868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:07.677658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 30/61] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:11:07 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:10.796793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9248f-0349-4316-a65f-1e648d473b22 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:10.797292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9248f-0349-4316-a65f-1e648d473b22 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:10.797365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9248f-0349-4316-a65f-1e648d473b22 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:10.798705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 32/62] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:10.817032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8b549-0d12-45c9-8244-cf410fbf5d11 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:10.817516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8b549-0d12-45c9-8244-cf410fbf5d11 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:10.817586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8b549-0d12-45c9-8244-cf410fbf5d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:10.839759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b7f8b549-0d12-45c9-8244-cf410fbf5d11 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:10.840960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b7f8b549-0d12-45c9-8244-cf410fbf5d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:10.931255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 31/63] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:10.966610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9248f-0349-4316-a65f-1e648d473b22 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:11.118080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9248f-0349-4316-a65f-1e648d473b22 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:11.170973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589bfa2-0bf3-4ba2-a104-7d6d4f7510dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:11.171804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589bfa2-0bf3-4ba2-a104-7d6d4f7510dc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.172091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589bfa2-0bf3-4ba2-a104-7d6d4f7510dc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.172371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589bfa2-0bf3-4ba2-a104-7d6d4f7510dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.192387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589bfa2-0bf3-4ba2-a104-7d6d4f7510dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c589bfa2-0bf3-4ba2-a104-7d6d4f7510dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:11.202315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:11.203488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:11.204261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 33/64] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:11.211008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8b549-0d12-45c9-8244-cf410fbf5d11 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:11.235666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9acb51-8684-45a5-a198-7390682a6d10 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.236255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9acb51-8684-45a5-a198-7390682a6d10 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.236716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9acb51-8684-45a5-a198-7390682a6d10 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.238416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 34/65] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:11:11.252382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c6c34-2f17-47f4-b0fa-e15fa48d859b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.253004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c6c34-2f17-47f4-b0fa-e15fa48d859b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.253482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c6c34-2f17-47f4-b0fa-e15fa48d859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.284515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-080c6c34-2f17-47f4-b0fa-e15fa48d859b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:11.285749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-080c6c34-2f17-47f4-b0fa-e15fa48d859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:11.342292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:11.402318 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8b549-0d12-45c9-8244-cf410fbf5d11 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:11.413873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 35/66] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:11.440855 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b483b3-73e6-421e-af0c-fbcd664fc473 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.441571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b483b3-73e6-421e-af0c-fbcd664fc473 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.441883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b483b3-73e6-421e-af0c-fbcd664fc473 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.443373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 36/67] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:11:11.444372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b483b3-73e6-421e-af0c-fbcd664fc473 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:11.454377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:11.462051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f3f08-30bd-4798-ae56-f0eee678747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:11.462956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f3f08-30bd-4798-ae56-f0eee678747f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.463422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f3f08-30bd-4798-ae56-f0eee678747f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.463842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f3f08-30bd-4798-ae56-f0eee678747f None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.465462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f3f08-30bd-4798-ae56-f0eee678747f None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:11.465921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f3f08-30bd-4798-ae56-f0eee678747f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:11.467614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f3f08-30bd-4798-ae56-f0eee678747f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:11.477683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 32/68] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:11.479407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df208b-3fa9-4ff8-8253-791fa31fb05e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.479910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df208b-3fa9-4ff8-8253-791fa31fb05e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.480881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df208b-3fa9-4ff8-8253-791fa31fb05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.497849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f0df208b-3fa9-4ff8-8253-791fa31fb05e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:11.499029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f0df208b-3fa9-4ff8-8253-791fa31fb05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:11.559730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b483b3-73e6-421e-af0c-fbcd664fc473 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:11.567718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 33/69] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:11.569194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0855257-9d75-4793-a1f2-1dfbb5916c0b None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.569492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0855257-9d75-4793-a1f2-1dfbb5916c0b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.569773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0855257-9d75-4793-a1f2-1dfbb5916c0b None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.571048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 34/70] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:11.575482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0855257-9d75-4793-a1f2-1dfbb5916c0b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:11.609474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb51854-c8e7-468d-8855-293292a0c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:11.610283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51854-c8e7-468d-8855-293292a0c51a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.610568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51854-c8e7-468d-8855-293292a0c51a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.610846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51854-c8e7-468d-8855-293292a0c51a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.621374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb51854-c8e7-468d-8855-293292a0c51a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-fcb51854-c8e7-468d-8855-293292a0c51a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:11.629287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:11.630135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:11.630879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 37/71] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:11.632128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09df4f-0446-4471-9733-3a1c9aead0cd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.632128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09df4f-0446-4471-9733-3a1c9aead0cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.632241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09df4f-0446-4471-9733-3a1c9aead0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.649188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8e09df4f-0446-4471-9733-3a1c9aead0cd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:11.650368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e09df4f-0446-4471-9733-3a1c9aead0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:11.676668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:11.699982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0855257-9d75-4793-a1f2-1dfbb5916c0b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:11.725481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:11.736891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 38/72] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:11.746685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936a9ed-94e0-4546-bb54-d7e7cf8308f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:11.746685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936a9ed-94e0-4546-bb54-d7e7cf8308f5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.746685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936a9ed-94e0-4546-bb54-d7e7cf8308f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.746685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936a9ed-94e0-4546-bb54-d7e7cf8308f5 None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.746987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09df4f-0446-4471-9733-3a1c9aead0cd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:11.756260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936a9ed-94e0-4546-bb54-d7e7cf8308f5 None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:11.757773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936a9ed-94e0-4546-bb54-d7e7cf8308f5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:11.759778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 35/73] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:11 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:11.760494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936a9ed-94e0-4546-bb54-d7e7cf8308f5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:11.866054 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09df4f-0446-4471-9733-3a1c9aead0cd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:11.877127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936a9ed-94e0-4546-bb54-d7e7cf8308f5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:11.899874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b545daa-7217-4242-a015-90c968fbc6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:11.900691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545daa-7217-4242-a015-90c968fbc6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.900975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545daa-7217-4242-a015-90c968fbc6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.901251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545daa-7217-4242-a015-90c968fbc6c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.910653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:11.911319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:11.911641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:11.911921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:11.912008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b545daa-7217-4242-a015-90c968fbc6c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:11.912674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:11.912985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:11.914373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8b545daa-7217-4242-a015-90c968fbc6c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:11.920690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:11.921533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:11.922288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 39/74] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:11.922922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b545daa-7217-4242-a015-90c968fbc6c7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:11.922984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 36/75] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:11.923845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:11.944557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:11.970106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.026093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b545daa-7217-4242-a015-90c968fbc6c7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.026550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a713-62cb-42ac-b9f2-37d23850bff1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.059692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f79b0-507a-4309-bbbc-9c8980709119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.060481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f79b0-507a-4309-bbbc-9c8980709119 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.060723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9b25d-52fa-4d58-bec0-51778a31cb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.060826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f79b0-507a-4309-bbbc-9c8980709119 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.061027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f79b0-507a-4309-bbbc-9c8980709119 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.061382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b25d-52fa-4d58-bec0-51778a31cb06 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.061673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b25d-52fa-4d58-bec0-51778a31cb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.061959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b25d-52fa-4d58-bec0-51778a31cb06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.071629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f79b0-507a-4309-bbbc-9c8980709119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-4f4f79b0-507a-4309-bbbc-9c8980709119 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:12.080287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:12.081235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 37/76] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:12.083544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9126ffd-ac31-4061-9e08-ccfddde8118d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.084232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9126ffd-ac31-4061-9e08-ccfddde8118d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.084232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9126ffd-ac31-4061-9e08-ccfddde8118d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.085007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 38/77] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:12.085355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9126ffd-ac31-4061-9e08-ccfddde8118d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.150881 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9b25d-52fa-4d58-bec0-51778a31cb06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.153197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9b25d-52fa-4d58-bec0-51778a31cb06 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.171657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.197308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9126ffd-ac31-4061-9e08-ccfddde8118d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.200348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.239034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389adb69-1964-482a-ba58-005741517559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.239856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389adb69-1964-482a-ba58-005741517559 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.240216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389adb69-1964-482a-ba58-005741517559 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.240570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389adb69-1964-482a-ba58-005741517559 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.241397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389adb69-1964-482a-ba58-005741517559 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:12.241777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389adb69-1964-482a-ba58-005741517559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.243239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389adb69-1964-482a-ba58-005741517559 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.252177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 39/78] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:12.253086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389adb69-1964-482a-ba58-005741517559 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.271314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 40/79] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:12.272240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9b25d-52fa-4d58-bec0-51778a31cb06 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.358082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389adb69-1964-482a-ba58-005741517559 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9b25d-52fa-4d58-bec0-51778a31cb06 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.401081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de39a07-4626-43c4-bd63-44761cb69f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.402201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39a07-4626-43c4-bd63-44761cb69f04 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.402915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39a07-4626-43c4-bd63-44761cb69f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.403364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39a07-4626-43c4-bd63-44761cb69f04 None admin] PATH_INFO: `/v3/users/04bc439d29964fc096fdffcc884a8f66` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.404211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.405482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de39a07-4626-43c4-bd63-44761cb69f04 None admin] RBAC: Authorizing `identity:get_user(user_id=04bc439d29964fc096fdffcc884a8f66)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.406983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de39a07-4626-43c4-bd63-44761cb69f04 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.408735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 40/80] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/users/04bc439d29964fc096fdffcc884a8f66 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:12.410920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a75d-244f-4795-8a54-7133cbbfc4ee None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.411434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a75d-244f-4795-8a54-7133cbbfc4ee None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.411908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a75d-244f-4795-8a54-7133cbbfc4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.426287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.426989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.427271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.427579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.428387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:12.428701 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.429932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b3e4a75d-244f-4795-8a54-7133cbbfc4ee None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:12.430169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.431251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b3e4a75d-244f-4795-8a54-7133cbbfc4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:12.443877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 41/81] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:12.443877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.465046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.507415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 41/82] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:12.508261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4a75d-244f-4795-8a54-7133cbbfc4ee None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.541151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98481b8-c148-4b7c-9752-b1ca70ff36d1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.575885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40367-6511-4930-9b43-967fc9fefe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.576768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40367-6511-4930-9b43-967fc9fefe73 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.577056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40367-6511-4930-9b43-967fc9fefe73 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.577336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40367-6511-4930-9b43-967fc9fefe73 None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.578784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd40367-6511-4930-9b43-967fc9fefe73 None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.580268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd40367-6511-4930-9b43-967fc9fefe73 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.581904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 42/83] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:12.586453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40367-6511-4930-9b43-967fc9fefe73 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.616884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4a75d-244f-4795-8a54-7133cbbfc4ee None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.633586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.650961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bbd51-8b94-48ad-9556-c0ea5ca733fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.652402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bbd51-8b94-48ad-9556-c0ea5ca733fd None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.652682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bbd51-8b94-48ad-9556-c0ea5ca733fd None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.653146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bbd51-8b94-48ad-9556-c0ea5ca733fd None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.654150 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bbd51-8b94-48ad-9556-c0ea5ca733fd None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.655646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bbd51-8b94-48ad-9556-c0ea5ca733fd None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.657550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 42/84] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:12.658125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bbd51-8b94-48ad-9556-c0ea5ca733fd None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.688110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40367-6511-4930-9b43-967fc9fefe73 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.691160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.726249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1e8c2-2b94-49f2-8fc8-7cafd8c1e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.726913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e8c2-2b94-49f2-8fc8-7cafd8c1e89e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.727202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e8c2-2b94-49f2-8fc8-7cafd8c1e89e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.727518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e8c2-2b94-49f2-8fc8-7cafd8c1e89e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.763368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bbd51-8b94-48ad-9556-c0ea5ca733fd None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.794666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1e8c2-2b94-49f2-8fc8-7cafd8c1e89e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.796171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1e8c2-2b94-49f2-8fc8-7cafd8c1e89e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.796717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70294fc7-ea45-45ff-b6ee-45c7cabf9105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.797591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294fc7-ea45-45ff-b6ee-45c7cabf9105 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.797880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294fc7-ea45-45ff-b6ee-45c7cabf9105 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.798159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294fc7-ea45-45ff-b6ee-45c7cabf9105 None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.799407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70294fc7-ea45-45ff-b6ee-45c7cabf9105 None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.800852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70294fc7-ea45-45ff-b6ee-45c7cabf9105 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.802501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 43/85] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:12.806921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70294fc7-ea45-45ff-b6ee-45c7cabf9105 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.846013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.897622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 43/86] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:12.898459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1e8c2-2b94-49f2-8fc8-7cafd8c1e89e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.900074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:12.904738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70294fc7-ea45-45ff-b6ee-45c7cabf9105 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:12.939138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8456a3-30b7-405c-8fa6-9d43de084bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:12.939998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8456a3-30b7-405c-8fa6-9d43de084bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:12.940284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8456a3-30b7-405c-8fa6-9d43de084bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:12.940556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8456a3-30b7-405c-8fa6-9d43de084bd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:12.941724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8456a3-30b7-405c-8fa6-9d43de084bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:12.943171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8456a3-30b7-405c-8fa6-9d43de084bd7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:12.944812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 44/87] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:12.945575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8456a3-30b7-405c-8fa6-9d43de084bd7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:12.997086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1e8c2-2b94-49f2-8fc8-7cafd8c1e89e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.030526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603e1c-dbb4-457e-ab77-d4890e667f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.031170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603e1c-dbb4-457e-ab77-d4890e667f3c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.031484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603e1c-dbb4-457e-ab77-d4890e667f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.031762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603e1c-dbb4-457e-ab77-d4890e667f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.044749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8456a3-30b7-405c-8fa6-9d43de084bd7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.049430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:13.079229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.079969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.080255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.080536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.081380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] RBAC: Adding query filter params (role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=7b05c4db805c48ef88067735bdedc47e, user.id=61bb099d65764401bb3305bd6d0703a6) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:13.081701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.083149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:13.088720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 45/88] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/role_assignments?user.id=61bb099d65764401bb3305bd6d0703a6&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:13.089391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:13.103505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603e1c-dbb4-457e-ab77-d4890e667f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.105099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603e1c-dbb4-457e-ab77-d4890e667f3c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:13.124613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:13.212189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e10bf-888c-4397-8df2-83e6ce8b01ba None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.220712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 44/89] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:13.221665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603e1c-dbb4-457e-ab77-d4890e667f3c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:13.256665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096d3b5-227f-418d-b0bd-30b7861f2f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.257376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096d3b5-227f-418d-b0bd-30b7861f2f26 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.257653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096d3b5-227f-418d-b0bd-30b7861f2f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.257942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096d3b5-227f-418d-b0bd-30b7861f2f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.280721 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:13.336317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096d3b5-227f-418d-b0bd-30b7861f2f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.337798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096d3b5-227f-418d-b0bd-30b7861f2f26 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:13.340011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603e1c-dbb4-457e-ab77-d4890e667f3c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.368801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:13.378074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941273b-190e-4b06-bc7d-98c241926236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.378954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941273b-190e-4b06-bc7d-98c241926236 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.379240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941273b-190e-4b06-bc7d-98c241926236 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.379543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941273b-190e-4b06-bc7d-98c241926236 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.381068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941273b-190e-4b06-bc7d-98c241926236 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.382526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941273b-190e-4b06-bc7d-98c241926236 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:13.384266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 45/90] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:13 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:13.389738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941273b-190e-4b06-bc7d-98c241926236 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:13.451451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 46/91] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:13.456904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096d3b5-227f-418d-b0bd-30b7861f2f26 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:13.499050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941273b-190e-4b06-bc7d-98c241926236 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.517525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:13.537187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e26a9f-bbb7-4c3f-a007-b9d1121e033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.537867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e26a9f-bbb7-4c3f-a007-b9d1121e033d None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.538154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e26a9f-bbb7-4c3f-a007-b9d1121e033d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.538433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e26a9f-bbb7-4c3f-a007-b9d1121e033d None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.539150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e26a9f-bbb7-4c3f-a007-b9d1121e033d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.540628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e26a9f-bbb7-4c3f-a007-b9d1121e033d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:13.566309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096d3b5-227f-418d-b0bd-30b7861f2f26 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.567820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 46/92] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:11:13 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:13.567820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e26a9f-bbb7-4c3f-a007-b9d1121e033d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:13.623550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:13.631969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a12599-7318-4d27-a295-30bdb91ab96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.633836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12599-7318-4d27-a295-30bdb91ab96e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.634348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12599-7318-4d27-a295-30bdb91ab96e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.634723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12599-7318-4d27-a295-30bdb91ab96e None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.638105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a12599-7318-4d27-a295-30bdb91ab96e None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.640430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a12599-7318-4d27-a295-30bdb91ab96e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:13.647530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 47/93] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:13 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:13.648462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a12599-7318-4d27-a295-30bdb91ab96e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:13.712462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e26a9f-bbb7-4c3f-a007-b9d1121e033d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.758582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.758582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.758582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.758969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.760904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=7b05c4db805c48ef88067735bdedc47e, user.id=04bc439d29964fc096fdffcc884a8f66) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:13.760904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.762961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a12599-7318-4d27-a295-30bdb91ab96e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.765643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:13.775184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 47/94] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:13 2022] GET /identity/v3/role_assignments?user.id=04bc439d29964fc096fdffcc884a8f66&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=e07cc5009f624eb9912a837c934e68aa => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:13.776070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:13.805997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768140c8-9957-4369-a130-8ed71db6c855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.806891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768140c8-9957-4369-a130-8ed71db6c855 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.807184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768140c8-9957-4369-a130-8ed71db6c855 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.807498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768140c8-9957-4369-a130-8ed71db6c855 None admin] PATH_INFO: `/v3/projects/c6b73840dac34cbd93caee1aefbf1379` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.809301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768140c8-9957-4369-a130-8ed71db6c855 None admin] RBAC: Authorizing `identity:get_project(project_id=c6b73840dac34cbd93caee1aefbf1379)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.810806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768140c8-9957-4369-a130-8ed71db6c855 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:13.814011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 48/95] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:13 2022] GET /identity/v3/projects/c6b73840dac34cbd93caee1aefbf1379 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:13.820021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768140c8-9957-4369-a130-8ed71db6c855 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:13.828896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:13.920391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:13.936914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cce8a-68b1-4189-a7f9-512b57fefbcc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.973874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768140c8-9957-4369-a130-8ed71db6c855 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:13.988930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d31e0-1851-4f65-ae4a-bdd294285494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:13.989607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d31e0-1851-4f65-ae4a-bdd294285494 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:13.989890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d31e0-1851-4f65-ae4a-bdd294285494 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:13.990175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d31e0-1851-4f65-ae4a-bdd294285494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:13.991026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d31e0-1851-4f65-ae4a-bdd294285494 None admin] RBAC: Adding query filter params (role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=7b05c4db805c48ef88067735bdedc47e, user.id=61bb099d65764401bb3305bd6d0703a6) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:13.991359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d31e0-1851-4f65-ae4a-bdd294285494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:13.992878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d31e0-1851-4f65-ae4a-bdd294285494 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:14.000400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 48/96] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:13 2022] GET /identity/v3/role_assignments?user.id=61bb099d65764401bb3305bd6d0703a6&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:14.023397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b5ed7-0602-42bf-93e9-05e2ac877fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:14.024090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b5ed7-0602-42bf-93e9-05e2ac877fcb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:14.024377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b5ed7-0602-42bf-93e9-05e2ac877fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:14.024655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b5ed7-0602-42bf-93e9-05e2ac877fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:14.025596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b5ed7-0602-42bf-93e9-05e2ac877fcb None admin] RBAC: Adding query filter params (role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=c6b73840dac34cbd93caee1aefbf1379, user.id=61bb099d65764401bb3305bd6d0703a6) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:14.025862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b5ed7-0602-42bf-93e9-05e2ac877fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:14.029621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b5ed7-0602-42bf-93e9-05e2ac877fcb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:14.034663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 49/97] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:13 2022] GET /identity/v3/role_assignments?user.id=61bb099d65764401bb3305bd6d0703a6&scope.project.id=c6b73840dac34cbd93caee1aefbf1379&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:15.809227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff4955-48ce-4462-a16f-462545e8767f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:15.810233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff4955-48ce-4462-a16f-462545e8767f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:15.810718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff4955-48ce-4462-a16f-462545e8767f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:15.812333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 49/98] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:15.817745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4df6c7-978c-4108-b54d-f6859eeeffb6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:15.818551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4df6c7-978c-4108-b54d-f6859eeeffb6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:15.819043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4df6c7-978c-4108-b54d-f6859eeeffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:15.843969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6c4df6c7-978c-4108-b54d-f6859eeeffb6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:15.845519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6c4df6c7-978c-4108-b54d-f6859eeeffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:15.928838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 50/99] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:15.962529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff4955-48ce-4462-a16f-462545e8767f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:16.080371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:16.084547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff4955-48ce-4462-a16f-462545e8767f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:16.125727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee399c-8699-46ed-8912-c7f296ebf0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:16.126537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee399c-8699-46ed-8912-c7f296ebf0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:16.126823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee399c-8699-46ed-8912-c7f296ebf0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:16.127098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee399c-8699-46ed-8912-c7f296ebf0f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:16.147121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee399c-8699-46ed-8912-c7f296ebf0f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-efee399c-8699-46ed-8912-c7f296ebf0f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:16.155725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:16.156658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:16.157436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 50/100] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:16.162899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4df6c7-978c-4108-b54d-f6859eeeffb6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:16.283210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:16.339249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4df6c7-978c-4108-b54d-f6859eeeffb6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:16.390879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:16.391586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:16.391873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:16.392156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:16.392922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:16.393236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:16.394632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:16.407994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 51/101] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:16.413656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee399c-8699-46ed-8912-c7f296ebf0f1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:16.561068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee399c-8699-46ed-8912-c7f296ebf0f1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:16.609931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2be31-dfcf-4348-bf5b-cf2170304027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:16.610745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2be31-dfcf-4348-bf5b-cf2170304027 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:16.611029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2be31-dfcf-4348-bf5b-cf2170304027 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:16.611306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2be31-dfcf-4348-bf5b-cf2170304027 None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:16.612575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2be31-dfcf-4348-bf5b-cf2170304027 None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:16.614039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2be31-dfcf-4348-bf5b-cf2170304027 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:16.615900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 51/102] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:16 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:16.622080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:16.760239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4d0f1-6a39-4b5e-bf74-1c2bff97cead None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:16.793109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:16.805987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c11e56-36b0-4127-a66e-3d2f3fa6dda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:16.806862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11e56-36b0-4127-a66e-3d2f3fa6dda4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:16.807147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11e56-36b0-4127-a66e-3d2f3fa6dda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:16.807449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11e56-36b0-4127-a66e-3d2f3fa6dda4 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:16.812068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11e56-36b0-4127-a66e-3d2f3fa6dda4 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:16.813537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11e56-36b0-4127-a66e-3d2f3fa6dda4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:16.815590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 52/103] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:16 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:16.821138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2be31-dfcf-4348-bf5b-cf2170304027 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:16.836253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2fd0e-6199-4f9a-9272-3762731f41bb None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:16.836767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2fd0e-6199-4f9a-9272-3762731f41bb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:16.836848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2fd0e-6199-4f9a-9272-3762731f41bb None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:16.838317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 53/104] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:16.849028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7455-c74c-4084-b17e-d4ecaf400176 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:16.849325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7455-c74c-4084-b17e-d4ecaf400176 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:16.849593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7455-c74c-4084-b17e-d4ecaf400176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:16.868952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:16.874972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5f6f7455-c74c-4084-b17e-d4ecaf400176 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:16.874972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5f6f7455-c74c-4084-b17e-d4ecaf400176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:17.020720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2be31-dfcf-4348-bf5b-cf2170304027 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:17.027864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 54/105] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:17.092750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be5748-0ff5-4add-ade6-7ba4d788b34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:17.092947 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f7455-c74c-4084-b17e-d4ecaf400176 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:17.093655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5748-0ff5-4add-ade6-7ba4d788b34a None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.093943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5748-0ff5-4add-ade6-7ba4d788b34a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.094221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5748-0ff5-4add-ade6-7ba4d788b34a None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e/users/61bb099d65764401bb3305bd6d0703a6/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.100885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be5748-0ff5-4add-ade6-7ba4d788b34a None admin] RBAC: Authorizing `identity:create_grant(project_id=7b05c4db805c48ef88067735bdedc47e, user_id=61bb099d65764401bb3305bd6d0703a6, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:17.102376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be5748-0ff5-4add-ade6-7ba4d788b34a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:17.131454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 52/106] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:16 2022] PUT /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e/users/61bb099d65764401bb3305bd6d0703a6/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:17.133854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0a2cf-1816-4689-a1e7-b4bdf4ddb068 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.134151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0a2cf-1816-4689-a1e7-b4bdf4ddb068 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.134443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0a2cf-1816-4689-a1e7-b4bdf4ddb068 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.140500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 53/107] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:17.177838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9373b4-25cd-4097-a327-500863b88227 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.178177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9373b4-25cd-4097-a327-500863b88227 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.178413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9373b4-25cd-4097-a327-500863b88227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.211006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3a9373b4-25cd-4097-a327-500863b88227 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:17.212295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3a9373b4-25cd-4097-a327-500863b88227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:17.308191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f7455-c74c-4084-b17e-d4ecaf400176 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:17.362453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 54/108] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:17.362909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:17.389942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11eb5e5-aec3-4939-9a73-63a48d4a20fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:17.390752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb5e5-aec3-4939-9a73-63a48d4a20fb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.391092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb5e5-aec3-4939-9a73-63a48d4a20fb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.391403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb5e5-aec3-4939-9a73-63a48d4a20fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.407867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11eb5e5-aec3-4939-9a73-63a48d4a20fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c11eb5e5-aec3-4939-9a73-63a48d4a20fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:17.419328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:17.420293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:17.421048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:17.421638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 55/109] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:17.426753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9373b4-25cd-4097-a327-500863b88227 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:17.429183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a691-4ce2-4ffc-a011-28f4793c46d0 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.429699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a691-4ce2-4ffc-a011-28f4793c46d0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.429924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a691-4ce2-4ffc-a011-28f4793c46d0 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.431643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 56/110] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:17.432450 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10a691-4ce2-4ffc-a011-28f4793c46d0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:17.467724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:17.569995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10a691-4ce2-4ffc-a011-28f4793c46d0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:17.586274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9373b4-25cd-4097-a327-500863b88227 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:17.633916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fac02-504a-4ddc-b64f-1e8eb556199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:17.635209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fac02-504a-4ddc-b64f-1e8eb556199a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.635534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fac02-504a-4ddc-b64f-1e8eb556199a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.636752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fac02-504a-4ddc-b64f-1e8eb556199a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.640949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cd3d2-eaea-4d70-a7d1-b0dfb8691709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:17.641576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cd3d2-eaea-4d70-a7d1-b0dfb8691709 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.641864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cd3d2-eaea-4d70-a7d1-b0dfb8691709 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.642145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cd3d2-eaea-4d70-a7d1-b0dfb8691709 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.643218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cd3d2-eaea-4d70-a7d1-b0dfb8691709 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:17.643567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cd3d2-eaea-4d70-a7d1-b0dfb8691709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:17.645259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cd3d2-eaea-4d70-a7d1-b0dfb8691709 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:17.648883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fac02-504a-4ddc-b64f-1e8eb556199a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:17.654949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 55/111] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:17.656442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050e910-5135-425f-88bf-7969de411587 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.656742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050e910-5135-425f-88bf-7969de411587 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.657027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050e910-5135-425f-88bf-7969de411587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-fd0fac02-504a-4ddc-b64f-1e8eb556199a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:17.659208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:17.660090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:17.660847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:17.661442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 57/112] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:17.668270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5ea60-a10e-44aa-ad0e-2c2fdf26a302 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.668270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5ea60-a10e-44aa-ad0e-2c2fdf26a302 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.668270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5ea60-a10e-44aa-ad0e-2c2fdf26a302 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.668270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 58/113] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:17.668270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5ea60-a10e-44aa-ad0e-2c2fdf26a302 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:17.684782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7050e910-5135-425f-88bf-7969de411587 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:17.686560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7050e910-5135-425f-88bf-7969de411587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:17.730965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:17.778956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 56/114] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:17.785582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050e910-5135-425f-88bf-7969de411587 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:17.806047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:17.817904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5ea60-a10e-44aa-ad0e-2c2fdf26a302 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:17.856719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604bcb4-06d6-41a3-824e-56a3b0e9c9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:17.857757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604bcb4-06d6-41a3-824e-56a3b0e9c9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.858293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604bcb4-06d6-41a3-824e-56a3b0e9c9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.858575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604bcb4-06d6-41a3-824e-56a3b0e9c9e5 None admin] PATH_INFO: `/v3/users/aeb45785ff2545a8b0c9e8c838712dcc` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.873338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604bcb4-06d6-41a3-824e-56a3b0e9c9e5 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb45785ff2545a8b0c9e8c838712dcc)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:17.874844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604bcb4-06d6-41a3-824e-56a3b0e9c9e5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:17.878004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 59/115] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:17 2022] GET /identity/v3/users/aeb45785ff2545a8b0c9e8c838712dcc => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:17.878004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b0742-3a24-4fd2-b3c5-7b822d24b3d4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.878317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b0742-3a24-4fd2-b3c5-7b822d24b3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.878610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b0742-3a24-4fd2-b3c5-7b822d24b3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.896749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f37b0742-3a24-4fd2-b3c5-7b822d24b3d4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:17.897919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f37b0742-3a24-4fd2-b3c5-7b822d24b3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:17.902666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050e910-5135-425f-88bf-7969de411587 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:17.939494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cb322-dc38-4831-b280-284953f6da72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:17.940176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb322-dc38-4831-b280-284953f6da72 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.940462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb322-dc38-4831-b280-284953f6da72 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.940737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb322-dc38-4831-b280-284953f6da72 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.941496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cb322-dc38-4831-b280-284953f6da72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:17.941811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cb322-dc38-4831-b280-284953f6da72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:17.943219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cb322-dc38-4831-b280-284953f6da72 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:17.951622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 57/116] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:17.952913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca87b-3792-4214-99a6-6dfd0f815912 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:17.953213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca87b-3792-4214-99a6-6dfd0f815912 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:17.953499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca87b-3792-4214-99a6-6dfd0f815912 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:17.958751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 58/117] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:17.958751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca87b-3792-4214-99a6-6dfd0f815912 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:17.981431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 60/118] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:17.982208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b0742-3a24-4fd2-b3c5-7b822d24b3d4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.010626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.053615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.087813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b0742-3a24-4fd2-b3c5-7b822d24b3d4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.091014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca87b-3792-4214-99a6-6dfd0f815912 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.133193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492fe1a-c283-42d5-a3b1-3631faaa8f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.133851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492fe1a-c283-42d5-a3b1-3631faaa8f69 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.134133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492fe1a-c283-42d5-a3b1-3631faaa8f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.134409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492fe1a-c283-42d5-a3b1-3631faaa8f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.134726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba79f79-10ce-4ce6-b1dd-247e783f8f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.135565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79f79-10ce-4ce6-b1dd-247e783f8f33 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.135853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79f79-10ce-4ce6-b1dd-247e783f8f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.136709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79f79-10ce-4ce6-b1dd-247e783f8f33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.148181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba79f79-10ce-4ce6-b1dd-247e783f8f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-fba79f79-10ce-4ce6-b1dd-247e783f8f33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:18.155916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:18.156792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:18.157563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 59/119] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:18.158749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba79f79-10ce-4ce6-b1dd-247e783f8f33 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.217568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492fe1a-c283-42d5-a3b1-3631faaa8f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:18.219107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492fe1a-c283-42d5-a3b1-3631faaa8f69 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:18.247099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.270578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba79f79-10ce-4ce6-b1dd-247e783f8f33 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.302076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.307283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e274af0-7d5d-482d-b4cc-a7e07a262287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.309687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274af0-7d5d-482d-b4cc-a7e07a262287 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.310394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274af0-7d5d-482d-b4cc-a7e07a262287 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.310913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274af0-7d5d-482d-b4cc-a7e07a262287 None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.312738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e274af0-7d5d-482d-b4cc-a7e07a262287 None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:18.314732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e274af0-7d5d-482d-b4cc-a7e07a262287 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:18.317323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 60/120] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:18.319272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b037baf-7ede-47e2-96de-6f5086357d18 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.319734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b037baf-7ede-47e2-96de-6f5086357d18 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.320137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b037baf-7ede-47e2-96de-6f5086357d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.321269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 61/121] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:18.322400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492fe1a-c283-42d5-a3b1-3631faaa8f69 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.341321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9b037baf-7ede-47e2-96de-6f5086357d18 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:18.342514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9b037baf-7ede-47e2-96de-6f5086357d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:18.427564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 61/122] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:18.427564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b037baf-7ede-47e2-96de-6f5086357d18 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.446669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492fe1a-c283-42d5-a3b1-3631faaa8f69 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.475126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.484601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d1df1-d24b-4dd1-accf-70b3376f139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.485497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d1df1-d24b-4dd1-accf-70b3376f139c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.485837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d1df1-d24b-4dd1-accf-70b3376f139c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.486185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d1df1-d24b-4dd1-accf-70b3376f139c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.497707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d1df1-d24b-4dd1-accf-70b3376f139c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-447d1df1-d24b-4dd1-accf-70b3376f139c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:18.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:18.506670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 62/123] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:18.507667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d1df1-d24b-4dd1-accf-70b3376f139c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.554588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b037baf-7ede-47e2-96de-6f5086357d18 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.586466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.594545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.595238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.595558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.596200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.596960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:18.597585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:18.598965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:18.614441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 62/124] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:18.614441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.624820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d1df1-d24b-4dd1-accf-70b3376f139c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.659415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fc480-e503-4135-ad94-310a27971609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.660287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fc480-e503-4135-ad94-310a27971609 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.660576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fc480-e503-4135-ad94-310a27971609 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.660853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fc480-e503-4135-ad94-310a27971609 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.662058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fc480-e503-4135-ad94-310a27971609 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:18.663564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fc480-e503-4135-ad94-310a27971609 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:18.665474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 63/125] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:18.666049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fc480-e503-4135-ad94-310a27971609 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.724522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a654-1ff1-458a-8328-c060822dd029 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.732141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.765789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac289259-fd2b-4fd1-91db-e8bab8b11281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.766670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289259-fd2b-4fd1-91db-e8bab8b11281 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.766986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289259-fd2b-4fd1-91db-e8bab8b11281 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.767222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289259-fd2b-4fd1-91db-e8bab8b11281 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.768463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac289259-fd2b-4fd1-91db-e8bab8b11281 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:18.769943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac289259-fd2b-4fd1-91db-e8bab8b11281 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:18.772010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 63/126] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:18.772799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac289259-fd2b-4fd1-91db-e8bab8b11281 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.789081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fc480-e503-4135-ad94-310a27971609 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.827111 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2097ff2-dfb1-4660-baa7-280d6cc1e8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.827934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2097ff2-dfb1-4660-baa7-280d6cc1e8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.828219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2097ff2-dfb1-4660-baa7-280d6cc1e8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.828496 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2097ff2-dfb1-4660-baa7-280d6cc1e8e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.829734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2097ff2-dfb1-4660-baa7-280d6cc1e8e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:18.831184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2097ff2-dfb1-4660-baa7-280d6cc1e8e6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:18.833170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 64/127] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:18.833941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2097ff2-dfb1-4660-baa7-280d6cc1e8e6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.839915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.884982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac289259-fd2b-4fd1-91db-e8bab8b11281 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.919875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.920583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.920834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.921117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.921870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:18.922181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:18.923592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:18.936602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 64/128] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:18.936602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:18.944702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2097ff2-dfb1-4660-baa7-280d6cc1e8e6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:18.970646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:18.984707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ad3c3-382d-49a2-b8b2-d57322281fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:18.985526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad3c3-382d-49a2-b8b2-d57322281fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:18.985818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad3c3-382d-49a2-b8b2-d57322281fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:18.986095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad3c3-382d-49a2-b8b2-d57322281fc1 None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:18.987302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ad3c3-382d-49a2-b8b2-d57322281fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:18.988802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ad3c3-382d-49a2-b8b2-d57322281fc1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:18.990599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 65/129] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:18.996058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ad3c3-382d-49a2-b8b2-d57322281fc1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:19.037566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec0b5a-8360-4828-98cb-3a5c5ffd20f2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:19.061153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:19.073804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6fead-1f9c-4833-b8aa-85f9f39a961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:19.074756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6fead-1f9c-4833-b8aa-85f9f39a961b None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.075043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6fead-1f9c-4833-b8aa-85f9f39a961b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.075332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6fead-1f9c-4833-b8aa-85f9f39a961b None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e/users/61bb099d65764401bb3305bd6d0703a6/roles/64cbdf4e69fa400b8b2f8ed13ebe5367` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.077611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6fead-1f9c-4833-b8aa-85f9f39a961b None admin] RBAC: Authorizing `identity:create_grant(project_id=7b05c4db805c48ef88067735bdedc47e, user_id=61bb099d65764401bb3305bd6d0703a6, role_id=64cbdf4e69fa400b8b2f8ed13ebe5367)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:19.079071 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6fead-1f9c-4833-b8aa-85f9f39a961b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:19.103373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 65/130] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:18 2022] PUT /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e/users/61bb099d65764401bb3305bd6d0703a6/roles/64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:19.103373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6fead-1f9c-4833-b8aa-85f9f39a961b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:19.134861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ad3c3-382d-49a2-b8b2-d57322281fc1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:19.178310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:19.179018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.179364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.179662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.180573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] RBAC: Adding query filter params (role.id=7b21f490e8634fb09bf8b449b44a937a, scope.project.id=7b05c4db805c48ef88067735bdedc47e, user.id=aeb45785ff2545a8b0c9e8c838712dcc) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:19.180895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:19.182404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:19.190233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 66/131] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:18 2022] GET /identity/v3/role_assignments?user.id=aeb45785ff2545a8b0c9e8c838712dcc&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=7b21f490e8634fb09bf8b449b44a937a => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:19.191121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:19.280683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6fead-1f9c-4833-b8aa-85f9f39a961b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:19.314018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:19.339798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e829ef-814d-42da-ae7f-8a76495fa932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:19.341003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e829ef-814d-42da-ae7f-8a76495fa932 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.342058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e829ef-814d-42da-ae7f-8a76495fa932 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.342545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e829ef-814d-42da-ae7f-8a76495fa932 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.344116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e829ef-814d-42da-ae7f-8a76495fa932 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:19.346201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e829ef-814d-42da-ae7f-8a76495fa932 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:19.370330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df79846-16bd-4e74-9aeb-3488fa5a96b4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:19.385451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 66/132] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:11:19 2022] POST /identity/v3/projects => generated 302 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:19.387567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e829ef-814d-42da-ae7f-8a76495fa932 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:19.436922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b39fd3-9cfa-45df-bf7f-9098b325bd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:19.437898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b39fd3-9cfa-45df-bf7f-9098b325bd8e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.438183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b39fd3-9cfa-45df-bf7f-9098b325bd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.438506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b39fd3-9cfa-45df-bf7f-9098b325bd8e None admin] PATH_INFO: `/v3/users/04bc439d29964fc096fdffcc884a8f66` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.440891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b39fd3-9cfa-45df-bf7f-9098b325bd8e None admin] RBAC: Authorizing `identity:get_user(user_id=04bc439d29964fc096fdffcc884a8f66)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:19.442407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b39fd3-9cfa-45df-bf7f-9098b325bd8e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:19.444741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 67/133] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:19 2022] GET /identity/v3/users/04bc439d29964fc096fdffcc884a8f66 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:19.456525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b39fd3-9cfa-45df-bf7f-9098b325bd8e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:19.481907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:19.632090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e829ef-814d-42da-ae7f-8a76495fa932 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:19.693216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b39fd3-9cfa-45df-bf7f-9098b325bd8e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:19.704240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e98bb-0b8d-4964-b523-2dec278ef38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:19.705123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e98bb-0b8d-4964-b523-2dec278ef38e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.705407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e98bb-0b8d-4964-b523-2dec278ef38e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.705694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e98bb-0b8d-4964-b523-2dec278ef38e None admin] PATH_INFO: `/v3/projects/c6b73840dac34cbd93caee1aefbf1379` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.708685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e98bb-0b8d-4964-b523-2dec278ef38e None admin] RBAC: Authorizing `identity:get_project(project_id=c6b73840dac34cbd93caee1aefbf1379)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:19.710173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e98bb-0b8d-4964-b523-2dec278ef38e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:19.713283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 67/134] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:19 2022] GET /identity/v3/projects/c6b73840dac34cbd93caee1aefbf1379 => generated 312 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:19.724640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e98bb-0b8d-4964-b523-2dec278ef38e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:19.743865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:19.760806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eff1d0-b959-4377-8c0f-b140c3de7da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:19.762423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eff1d0-b959-4377-8c0f-b140c3de7da3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.763042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eff1d0-b959-4377-8c0f-b140c3de7da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.763355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eff1d0-b959-4377-8c0f-b140c3de7da3 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.764892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eff1d0-b959-4377-8c0f-b140c3de7da3 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:19.766847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eff1d0-b959-4377-8c0f-b140c3de7da3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:19.769082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 68/135] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:19 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:19.770191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43132262-ee15-4e3b-9804-0dd190faa4a6 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.770455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43132262-ee15-4e3b-9804-0dd190faa4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.772615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43132262-ee15-4e3b-9804-0dd190faa4a6 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.773923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 69/136] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:19.779222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43132262-ee15-4e3b-9804-0dd190faa4a6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:19.862500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e98bb-0b8d-4964-b523-2dec278ef38e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:19.864623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:19.924703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16100f9e-4f65-4a18-997b-02e0802e2294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:19.925598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16100f9e-4f65-4a18-997b-02e0802e2294 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.925884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16100f9e-4f65-4a18-997b-02e0802e2294 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.926161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16100f9e-4f65-4a18-997b-02e0802e2294 None admin] PATH_INFO: `/v3/projects/c6b73840dac34cbd93caee1aefbf1379/users/61bb099d65764401bb3305bd6d0703a6/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.928280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43132262-ee15-4e3b-9804-0dd190faa4a6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:19.930509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16100f9e-4f65-4a18-997b-02e0802e2294 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6b73840dac34cbd93caee1aefbf1379, user_id=61bb099d65764401bb3305bd6d0703a6, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:19.932091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16100f9e-4f65-4a18-997b-02e0802e2294 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:19.960507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 68/137] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:19 2022] PUT /identity/v3/projects/c6b73840dac34cbd93caee1aefbf1379/users/61bb099d65764401bb3305bd6d0703a6/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:11:19.961710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514ace-74a2-455b-8127-f21c131e683c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:19.962005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514ace-74a2-455b-8127-f21c131e683c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:19.962342 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514ace-74a2-455b-8127-f21c131e683c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:19.996517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9e514ace-74a2-455b-8127-f21c131e683c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:19.997762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9e514ace-74a2-455b-8127-f21c131e683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:20.005281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391487-74bd-47f3-9d4e-adf1415da344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:20.006258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391487-74bd-47f3-9d4e-adf1415da344 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:20.006546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391487-74bd-47f3-9d4e-adf1415da344 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:20.006826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391487-74bd-47f3-9d4e-adf1415da344 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e/users/04bc439d29964fc096fdffcc884a8f66/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:20.011746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84391487-74bd-47f3-9d4e-adf1415da344 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b05c4db805c48ef88067735bdedc47e, user_id=04bc439d29964fc096fdffcc884a8f66, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:20.015509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84391487-74bd-47f3-9d4e-adf1415da344 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:20.039299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 70/138] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:19 2022] PUT /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e/users/04bc439d29964fc096fdffcc884a8f66/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:20.103506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 69/139] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:20.127466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391487-74bd-47f3-9d4e-adf1415da344 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:20.201108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:20.269216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391487-74bd-47f3-9d4e-adf1415da344 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:20.307489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdd798-d7e7-4a7a-a109-bed97dda3e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:20.308619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdd798-d7e7-4a7a-a109-bed97dda3e65 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:20.308907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdd798-d7e7-4a7a-a109-bed97dda3e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:20.309187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdd798-d7e7-4a7a-a109-bed97dda3e65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:20.321228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdd798-d7e7-4a7a-a109-bed97dda3e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cbbdd798-d7e7-4a7a-a109-bed97dda3e65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:20.329784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:20.330713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:20.331517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 71/140] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:20.339927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514ace-74a2-455b-8127-f21c131e683c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:20.482142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:20.493461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514ace-74a2-455b-8127-f21c131e683c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:20.537971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:20.538661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:20.538947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:20.539229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:20.540029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:20.540343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:20.541790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:20.555143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 70/141] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:20.561324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdd798-d7e7-4a7a-a109-bed97dda3e65 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:20.726609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdd798-d7e7-4a7a-a109-bed97dda3e65 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:20.747082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:20.777577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6309-0aa7-4d5c-8aae-bba0f1d1c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:20.778490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6309-0aa7-4d5c-8aae-bba0f1d1c85b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:20.778792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6309-0aa7-4d5c-8aae-bba0f1d1c85b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:20.779094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6309-0aa7-4d5c-8aae-bba0f1d1c85b None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:20.780367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f6309-0aa7-4d5c-8aae-bba0f1d1c85b None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:20.781838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f6309-0aa7-4d5c-8aae-bba0f1d1c85b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:20.783515 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 72/142] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:20 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:20.789144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:20.939162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7306c-8cf4-4646-acbd-67ef96a85a0a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:20.985536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233b83d-173f-4e95-9157-3da1346e8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:20.986223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233b83d-173f-4e95-9157-3da1346e8312 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:20.986535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233b83d-173f-4e95-9157-3da1346e8312 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:20.986856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233b83d-173f-4e95-9157-3da1346e8312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:21.026106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:21.080707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233b83d-173f-4e95-9157-3da1346e8312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:21.082178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233b83d-173f-4e95-9157-3da1346e8312 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:21.228170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 71/143] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:21.234701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6309-0aa7-4d5c-8aae-bba0f1d1c85b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:21.365256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6309-0aa7-4d5c-8aae-bba0f1d1c85b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:21.416281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedc32d-d5cb-4e29-8c99-000c37397f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:21.417311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc32d-d5cb-4e29-8c99-000c37397f42 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:21.417604 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc32d-d5cb-4e29-8c99-000c37397f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:21.417886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc32d-d5cb-4e29-8c99-000c37397f42 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:21.419161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfedc32d-d5cb-4e29-8c99-000c37397f42 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:21.420697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfedc32d-d5cb-4e29-8c99-000c37397f42 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:21.422741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 73/144] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:21 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:21.432512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233b83d-173f-4e95-9157-3da1346e8312 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:21.530583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:21.591537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233b83d-173f-4e95-9157-3da1346e8312 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:21.641042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc3cab-b85b-43a3-9761-29961906fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:21.641694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3cab-b85b-43a3-9761-29961906fa65 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:21.641991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3cab-b85b-43a3-9761-29961906fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:21.642276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3cab-b85b-43a3-9761-29961906fa65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:21.643121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc3cab-b85b-43a3-9761-29961906fa65 None admin] RBAC: Adding query filter params (role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=7b05c4db805c48ef88067735bdedc47e, user.id=61bb099d65764401bb3305bd6d0703a6) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:21.643468 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc3cab-b85b-43a3-9761-29961906fa65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:21.644951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc3cab-b85b-43a3-9761-29961906fa65 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:21.652656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 72/145] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:21 2022] GET /identity/v3/role_assignments?user.id=61bb099d65764401bb3305bd6d0703a6&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:22.561155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd2b3-4613-4415-914f-c4dc808057e3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:22.561702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd2b3-4613-4415-914f-c4dc808057e3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:22.561702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd2b3-4613-4415-914f-c4dc808057e3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:22.563445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 74/146] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:22.568876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a06c3-8a7c-4fb8-b849-906ea182a82a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:22.569304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a06c3-8a7c-4fb8-b849-906ea182a82a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:22.569491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a06c3-8a7c-4fb8-b849-906ea182a82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:22.591848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-906a06c3-8a7c-4fb8-b849-906ea182a82a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:22.592321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f4e2b-bcac-404b-a48f-0972b37163bb None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:22.592615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f4e2b-bcac-404b-a48f-0972b37163bb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:22.592900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f4e2b-bcac-404b-a48f-0972b37163bb None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:22.593163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-906a06c3-8a7c-4fb8-b849-906ea182a82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:22.594472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 75/147] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:22.610343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892393-5c08-4920-8e9a-6e3515cadbd1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:22.610648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892393-5c08-4920-8e9a-6e3515cadbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:22.610917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892393-5c08-4920-8e9a-6e3515cadbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:22.649515 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-36892393-5c08-4920-8e9a-6e3515cadbd1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:22.650781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-36892393-5c08-4920-8e9a-6e3515cadbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:22.714557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 73/148] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:22.755780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:22.785784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a06c3-8a7c-4fb8-b849-906ea182a82a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:22.789896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 76/149] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:22.814794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892393-5c08-4920-8e9a-6e3515cadbd1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:22.854356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:22.916188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a06c3-8a7c-4fb8-b849-906ea182a82a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:22.952848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c5cda-f12b-49fd-a67a-d2f462550595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:22.953369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892393-5c08-4920-8e9a-6e3515cadbd1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:22.954235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c5cda-f12b-49fd-a67a-d2f462550595 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:22.954715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c5cda-f12b-49fd-a67a-d2f462550595 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:22.955184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c5cda-f12b-49fd-a67a-d2f462550595 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:22.956681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c5cda-f12b-49fd-a67a-d2f462550595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:22.958377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c5cda-f12b-49fd-a67a-d2f462550595 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:22.960619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 74/150] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:22.962125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe053e8-e602-4aaf-aa81-db6d87ea4af7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:22.962609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe053e8-e602-4aaf-aa81-db6d87ea4af7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:22.963150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe053e8-e602-4aaf-aa81-db6d87ea4af7 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:22.965170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 75/151] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:22.970956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd27d3-3b96-4098-ae93-f780cbb6add7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:22.971509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd27d3-3b96-4098-ae93-f780cbb6add7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:22.971984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd27d3-3b96-4098-ae93-f780cbb6add7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:22.994288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f6bd27d3-3b96-4098-ae93-f780cbb6add7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:22.997420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f6bd27d3-3b96-4098-ae93-f780cbb6add7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:22.997962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edfd20-4d1b-40e6-84a5-1383cec5e9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:22.998836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edfd20-4d1b-40e6-84a5-1383cec5e9dc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:22.999114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edfd20-4d1b-40e6-84a5-1383cec5e9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:22.999477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edfd20-4d1b-40e6-84a5-1383cec5e9dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.013798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edfd20-4d1b-40e6-84a5-1383cec5e9dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e0edfd20-4d1b-40e6-84a5-1383cec5e9dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:23.024884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:23.025732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:23.026572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 77/152] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:23.027209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edfd20-4d1b-40e6-84a5-1383cec5e9dc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:23.087309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:23.091143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 76/153] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:23.092480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd27d3-3b96-4098-ae93-f780cbb6add7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:23.164200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:23.183199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edfd20-4d1b-40e6-84a5-1383cec5e9dc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:23.226854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997f73-a10f-438b-893c-e3759fff7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:23.227560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997f73-a10f-438b-893c-e3759fff7667 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.227841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997f73-a10f-438b-893c-e3759fff7667 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.228115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997f73-a10f-438b-893c-e3759fff7667 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.229161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c997f73-a10f-438b-893c-e3759fff7667 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:23.231696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c997f73-a10f-438b-893c-e3759fff7667 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:23.234614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd27d3-3b96-4098-ae93-f780cbb6add7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:23.255963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 78/154] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:11:23 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:23.256789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997f73-a10f-438b-893c-e3759fff7667 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:23.277164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebea288-cbc7-4b9b-9c02-e1245d217b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:23.278230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebea288-cbc7-4b9b-9c02-e1245d217b4e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.278690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebea288-cbc7-4b9b-9c02-e1245d217b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.279202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebea288-cbc7-4b9b-9c02-e1245d217b4e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.280125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebea288-cbc7-4b9b-9c02-e1245d217b4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:23.280509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebea288-cbc7-4b9b-9c02-e1245d217b4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:23.282027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebea288-cbc7-4b9b-9c02-e1245d217b4e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:23.291926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 77/155] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:23.293804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdbe5a-ea91-4a14-895a-fc9c5dd675d8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.294162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdbe5a-ea91-4a14-895a-fc9c5dd675d8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.294513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdbe5a-ea91-4a14-895a-fc9c5dd675d8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.295884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 78/156] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:23.300833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdbe5a-ea91-4a14-895a-fc9c5dd675d8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:23.386029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:23.395584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997f73-a10f-438b-893c-e3759fff7667 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:23.417558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdbe5a-ea91-4a14-895a-fc9c5dd675d8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:23.428720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:23.435327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771727a-7b9e-4378-a3a2-3010485f81a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:23.436174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771727a-7b9e-4378-a3a2-3010485f81a6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.436511 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771727a-7b9e-4378-a3a2-3010485f81a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.436794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771727a-7b9e-4378-a3a2-3010485f81a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.452962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771727a-7b9e-4378-a3a2-3010485f81a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-4771727a-7b9e-4378-a3a2-3010485f81a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:23.459965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:23.461228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 79/157] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:23.462449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4d04-4140-4b27-bc74-37c796e5ae61 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.463313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4d04-4140-4b27-bc74-37c796e5ae61 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.463313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4d04-4140-4b27-bc74-37c796e5ae61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.469851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd00770-6f2b-48da-aa83-f4e1102f6494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:23.470923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd00770-6f2b-48da-aa83-f4e1102f6494 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.471029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd00770-6f2b-48da-aa83-f4e1102f6494 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.471986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd00770-6f2b-48da-aa83-f4e1102f6494 None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.473651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd00770-6f2b-48da-aa83-f4e1102f6494 None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:23.475050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd00770-6f2b-48da-aa83-f4e1102f6494 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:23.477046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 79/158] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:23 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:23.477926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c500ed-2e9c-471f-a0d5-8433a29abe51 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.478210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c500ed-2e9c-471f-a0d5-8433a29abe51 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.478484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c500ed-2e9c-471f-a0d5-8433a29abe51 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.483387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 80/159] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:23.483387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c500ed-2e9c-471f-a0d5-8433a29abe51 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:23.484777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-57ea4d04-4140-4b27-bc74-37c796e5ae61 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:23.486104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-57ea4d04-4140-4b27-bc74-37c796e5ae61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:23.570613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 80/160] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:23.571511 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea4d04-4140-4b27-bc74-37c796e5ae61 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:23.624697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c500ed-2e9c-471f-a0d5-8433a29abe51 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:23.665096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3290ba-d51e-4cef-b656-7c5981dad683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:23.665788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3290ba-d51e-4cef-b656-7c5981dad683 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.666144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3290ba-d51e-4cef-b656-7c5981dad683 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.666696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3290ba-d51e-4cef-b656-7c5981dad683 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.667383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3290ba-d51e-4cef-b656-7c5981dad683 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:23.667766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3290ba-d51e-4cef-b656-7c5981dad683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:23.669225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3290ba-d51e-4cef-b656-7c5981dad683 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:23.679382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 81/161] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:23.683407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:23.683564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377699a-b4e9-4045-984f-a569a3e9cbc3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.683564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377699a-b4e9-4045-984f-a569a3e9cbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.684019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377699a-b4e9-4045-984f-a569a3e9cbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.704448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea4d04-4140-4b27-bc74-37c796e5ae61 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:23.704741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f377699a-b4e9-4045-984f-a569a3e9cbc3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:23.705930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f377699a-b4e9-4045-984f-a569a3e9cbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:23.746458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:23.753120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e517cc1-f1a4-4a66-8a4f-8914e9478ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:23.753997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e517cc1-f1a4-4a66-8a4f-8914e9478ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.754420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e517cc1-f1a4-4a66-8a4f-8914e9478ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.754865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e517cc1-f1a4-4a66-8a4f-8914e9478ad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.800712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 82/162] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:23.801806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377699a-b4e9-4045-984f-a569a3e9cbc3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:23.861461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e517cc1-f1a4-4a66-8a4f-8914e9478ad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:23.864094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e517cc1-f1a4-4a66-8a4f-8914e9478ad2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:23.925839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377699a-b4e9-4045-984f-a569a3e9cbc3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:23.966522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03b95f-d1d3-4a4a-8835-38679817aef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:23.967394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03b95f-d1d3-4a4a-8835-38679817aef9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:23.967653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03b95f-d1d3-4a4a-8835-38679817aef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:23.967931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03b95f-d1d3-4a4a-8835-38679817aef9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:23.981137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03b95f-d1d3-4a4a-8835-38679817aef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:23.989817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ac03b95f-d1d3-4a4a-8835-38679817aef9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:23.990148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:23.991012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:23.991820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 83/163] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:23.992452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03b95f-d1d3-4a4a-8835-38679817aef9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:24.012836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 81/164] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:24.013752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e517cc1-f1a4-4a66-8a4f-8914e9478ad2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:24.025738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:24.163364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03b95f-d1d3-4a4a-8835-38679817aef9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:24.208094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e517cc1-f1a4-4a66-8a4f-8914e9478ad2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:24.234103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6069-681d-4801-8022-1a8524d7faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:24.235085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6069-681d-4801-8022-1a8524d7faf0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:24.235417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6069-681d-4801-8022-1a8524d7faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:24.235734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6069-681d-4801-8022-1a8524d7faf0 None admin] PATH_INFO: `/v3/users/aeb45785ff2545a8b0c9e8c838712dcc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:24.237857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6069-681d-4801-8022-1a8524d7faf0 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb45785ff2545a8b0c9e8c838712dcc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:24.239521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6069-681d-4801-8022-1a8524d7faf0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:24.241990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 84/165] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:23 2022] GET /identity/v3/users/aeb45785ff2545a8b0c9e8c838712dcc => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:24.243004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6069-681d-4801-8022-1a8524d7faf0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:24.283750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83370c9f-b106-4bcb-8772-5460084aeea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:24.284617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83370c9f-b106-4bcb-8772-5460084aeea4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:24.284918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83370c9f-b106-4bcb-8772-5460084aeea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:24.285224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83370c9f-b106-4bcb-8772-5460084aeea4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:24.312389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83370c9f-b106-4bcb-8772-5460084aeea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-83370c9f-b106-4bcb-8772-5460084aeea4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:24.320915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:24.321794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:24.322581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 82/166] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:24.324510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83370c9f-b106-4bcb-8772-5460084aeea4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:24.395778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:24.424950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:24.437412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6069-681d-4801-8022-1a8524d7faf0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:24.491299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83370c9f-b106-4bcb-8772-5460084aeea4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:24.505435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:24.506110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:24.506435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:24.506713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:24.507494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:24.507800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:24.510424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:24.524333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 85/167] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:24.524995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:24.558298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7d324-66e4-48ed-96b7-99441528ba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:24.558988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7d324-66e4-48ed-96b7-99441528ba48 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:24.559274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7d324-66e4-48ed-96b7-99441528ba48 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:24.559628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7d324-66e4-48ed-96b7-99441528ba48 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:24.562195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7d324-66e4-48ed-96b7-99441528ba48 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:24.564573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7d324-66e4-48ed-96b7-99441528ba48 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:24.568208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 83/168] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:24 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:24.570117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7d324-66e4-48ed-96b7-99441528ba48 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:24.708527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7d324-66e4-48ed-96b7-99441528ba48 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:24.723522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:24.731867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ee006-1372-4069-a62f-70b85d1d5c4d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:24.769044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:24.769643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:24.769926 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:24.770204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:24.770965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:24.771273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:24.772682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:24.775853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8131b7-4370-458a-abd6-c767fceca39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:24.776945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8131b7-4370-458a-abd6-c767fceca39b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:24.777225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8131b7-4370-458a-abd6-c767fceca39b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:24.777559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8131b7-4370-458a-abd6-c767fceca39b None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:24.778842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8131b7-4370-458a-abd6-c767fceca39b None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:24.780318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8131b7-4370-458a-abd6-c767fceca39b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:24.782200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 86/169] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:24 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:24.782782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8131b7-4370-458a-abd6-c767fceca39b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:24.797401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 84/170] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:24.798106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:24.841151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:24.972635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8131b7-4370-458a-abd6-c767fceca39b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:24.987546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c26724-2de0-4267-aa09-e9b97b41ff77 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:25.023887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:25.024558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:25.024841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:25.025115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:25.026035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] RBAC: Adding query filter params (role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=7b05c4db805c48ef88067735bdedc47e, user.id=61bb099d65764401bb3305bd6d0703a6) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:25.026861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:25.028563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:25.035758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 85/171] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:24 2022] GET /identity/v3/role_assignments?user.id=61bb099d65764401bb3305bd6d0703a6&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:25.036580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:25.037181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026bb61-478c-4671-a020-87fdc72e563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:25.038188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026bb61-478c-4671-a020-87fdc72e563d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:25.038472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026bb61-478c-4671-a020-87fdc72e563d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:25.038747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026bb61-478c-4671-a020-87fdc72e563d None admin] PATH_INFO: `/v3/users/04bc439d29964fc096fdffcc884a8f66` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:25.044373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026bb61-478c-4671-a020-87fdc72e563d None admin] RBAC: Authorizing `identity:get_user(user_id=04bc439d29964fc096fdffcc884a8f66)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:25.045862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026bb61-478c-4671-a020-87fdc72e563d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:25.047833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 87/172] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:24 2022] GET /identity/v3/users/04bc439d29964fc096fdffcc884a8f66 => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:25.048480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026bb61-478c-4671-a020-87fdc72e563d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:25.126760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:25.234542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f7b1-7185-482a-a6e9-cff32d48c330 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:25.274862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa38a-020f-4dd9-a73d-ecae570e98f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:25.275865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa38a-020f-4dd9-a73d-ecae570e98f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:25.276159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa38a-020f-4dd9-a73d-ecae570e98f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:25.276502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa38a-020f-4dd9-a73d-ecae570e98f7 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e/users/aeb45785ff2545a8b0c9e8c838712dcc/roles/7b21f490e8634fb09bf8b449b44a937a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:25.278590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aa38a-020f-4dd9-a73d-ecae570e98f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b05c4db805c48ef88067735bdedc47e, user_id=aeb45785ff2545a8b0c9e8c838712dcc, role_id=7b21f490e8634fb09bf8b449b44a937a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:25.280154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aa38a-020f-4dd9-a73d-ecae570e98f7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:25.310318 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 86/173] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:25 2022] PUT /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e/users/aeb45785ff2545a8b0c9e8c838712dcc/roles/7b21f490e8634fb09bf8b449b44a937a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:25.311167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa38a-020f-4dd9-a73d-ecae570e98f7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:25.345546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:25.356394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026bb61-478c-4671-a020-87fdc72e563d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:25.409798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229a34-ae87-4efe-9b8b-5f299f8c2ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:25.412179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229a34-ae87-4efe-9b8b-5f299f8c2ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:25.412701 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229a34-ae87-4efe-9b8b-5f299f8c2ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:25.414349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229a34-ae87-4efe-9b8b-5f299f8c2ae7 None admin] PATH_INFO: `/v3/users/61bb099d65764401bb3305bd6d0703a6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:25.416299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17229a34-ae87-4efe-9b8b-5f299f8c2ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=61bb099d65764401bb3305bd6d0703a6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:25.422005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17229a34-ae87-4efe-9b8b-5f299f8c2ae7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:25.426521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 88/174] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:25 2022] GET /identity/v3/users/61bb099d65764401bb3305bd6d0703a6 => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:25.432122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229a34-ae87-4efe-9b8b-5f299f8c2ae7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:25.602760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:25.611159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229a34-ae87-4efe-9b8b-5f299f8c2ae7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:25.625502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa38a-020f-4dd9-a73d-ecae570e98f7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:25.662385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3ceb-bfc5-4200-ba88-7c0a7601c442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:25.663435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3ceb-bfc5-4200-ba88-7c0a7601c442 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:25.663949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3ceb-bfc5-4200-ba88-7c0a7601c442 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:25.664446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3ceb-bfc5-4200-ba88-7c0a7601c442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:25.700720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19661d-1c43-469b-a2bc-c91f083c2843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:25.701450 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19661d-1c43-469b-a2bc-c91f083c2843 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:25.701775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19661d-1c43-469b-a2bc-c91f083c2843 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:25.702079 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19661d-1c43-469b-a2bc-c91f083c2843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:25.706091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:25.749459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e3ceb-bfc5-4200-ba88-7c0a7601c442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:25.751107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e3ceb-bfc5-4200-ba88-7c0a7601c442 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:25.815444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19661d-1c43-469b-a2bc-c91f083c2843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:25.817083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19661d-1c43-469b-a2bc-c91f083c2843 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:25.862359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 89/175] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:25.868053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3ceb-bfc5-4200-ba88-7c0a7601c442 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:25.937484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 87/176] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:25.939730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:25.943501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19661d-1c43-469b-a2bc-c91f083c2843 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:25.944705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:25.975496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3ceb-bfc5-4200-ba88-7c0a7601c442 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:26.016148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7085662-b854-4e84-88b7-6d97da3b379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:26.017117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7085662-b854-4e84-88b7-6d97da3b379d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.017403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7085662-b854-4e84-88b7-6d97da3b379d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.017678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7085662-b854-4e84-88b7-6d97da3b379d None admin] PATH_INFO: `/v3/projects/c6b73840dac34cbd93caee1aefbf1379` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.019854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7085662-b854-4e84-88b7-6d97da3b379d None admin] RBAC: Authorizing `identity:get_project(project_id=c6b73840dac34cbd93caee1aefbf1379)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:26.021312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7085662-b854-4e84-88b7-6d97da3b379d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:26.023260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 90/177] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:25 2022] GET /identity/v3/projects/c6b73840dac34cbd93caee1aefbf1379 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:26.036429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7085662-b854-4e84-88b7-6d97da3b379d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:26.063830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19661d-1c43-469b-a2bc-c91f083c2843 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:26.101495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4368b-2d91-48b9-ab78-9db3d5ea53be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:26.102657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4368b-2d91-48b9-ab78-9db3d5ea53be None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.103040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4368b-2d91-48b9-ab78-9db3d5ea53be None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.103443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4368b-2d91-48b9-ab78-9db3d5ea53be None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.105475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4368b-2d91-48b9-ab78-9db3d5ea53be None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:26.107440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4368b-2d91-48b9-ab78-9db3d5ea53be None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:26.109517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 88/178] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:25 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:26.116433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4368b-2d91-48b9-ab78-9db3d5ea53be None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:26.239761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:26.258135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4368b-2d91-48b9-ab78-9db3d5ea53be None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:26.326314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b533a8f-cb57-4fdc-b769-4ea38b51c36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:26.327149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b533a8f-cb57-4fdc-b769-4ea38b51c36c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.327600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b533a8f-cb57-4fdc-b769-4ea38b51c36c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.327973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b533a8f-cb57-4fdc-b769-4ea38b51c36c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.329154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b533a8f-cb57-4fdc-b769-4ea38b51c36c None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=7b05c4db805c48ef88067735bdedc47e, user.id=04bc439d29964fc096fdffcc884a8f66) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:26.329567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b533a8f-cb57-4fdc-b769-4ea38b51c36c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:26.332264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b533a8f-cb57-4fdc-b769-4ea38b51c36c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:26.348263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 89/179] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:26 2022] GET /identity/v3/role_assignments?user.id=04bc439d29964fc096fdffcc884a8f66&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=e07cc5009f624eb9912a837c934e68aa => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:26.372089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7085662-b854-4e84-88b7-6d97da3b379d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:26.409003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3931b-defe-4ed0-a567-acc95ec94a4d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.409286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3931b-defe-4ed0-a567-acc95ec94a4d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.409561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3931b-defe-4ed0-a567-acc95ec94a4d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.410941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 90/180] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:26.419769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:26.427138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089b22a-e617-4079-8057-2f797f42faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:26.427837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089b22a-e617-4079-8057-2f797f42faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.428119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089b22a-e617-4079-8057-2f797f42faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.428393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089b22a-e617-4079-8057-2f797f42faa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.428700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553493a-adeb-4743-8a68-e4a74832547e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.429010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553493a-adeb-4743-8a68-e4a74832547e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.429307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553493a-adeb-4743-8a68-e4a74832547e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.429409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089b22a-e617-4079-8057-2f797f42faa4 None admin] RBAC: Adding query filter params (role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=c6b73840dac34cbd93caee1aefbf1379, user.id=61bb099d65764401bb3305bd6d0703a6) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:26.429723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089b22a-e617-4079-8057-2f797f42faa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:26.431297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089b22a-e617-4079-8057-2f797f42faa4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:26.447597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 91/181] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:26 2022] GET /identity/v3/role_assignments?user.id=61bb099d65764401bb3305bd6d0703a6&scope.project.id=c6b73840dac34cbd93caee1aefbf1379&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 608 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:26.459570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5553493a-adeb-4743-8a68-e4a74832547e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:26.460907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5553493a-adeb-4743-8a68-e4a74832547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:26.581412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 91/182] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:26.608273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284c84-9e10-4219-bc38-5e8961e34e95 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.608589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284c84-9e10-4219-bc38-5e8961e34e95 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.608864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284c84-9e10-4219-bc38-5e8961e34e95 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.610537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 92/183] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:26.627378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553493a-adeb-4743-8a68-e4a74832547e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:26.631163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa600-0686-4302-8d61-49434badb4fe None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.631528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa600-0686-4302-8d61-49434badb4fe None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.631808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa600-0686-4302-8d61-49434badb4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.663287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e81fa600-0686-4302-8d61-49434badb4fe None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:26.664574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e81fa600-0686-4302-8d61-49434badb4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:26.747714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:26.750294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 93/184] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:26.790549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa600-0686-4302-8d61-49434badb4fe None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:26.798163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553493a-adeb-4743-8a68-e4a74832547e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:26.839153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004aa4a-60f5-428e-8a6e-aece78f92753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:26.840449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004aa4a-60f5-428e-8a6e-aece78f92753 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.840841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004aa4a-60f5-428e-8a6e-aece78f92753 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.841208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004aa4a-60f5-428e-8a6e-aece78f92753 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.853096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004aa4a-60f5-428e-8a6e-aece78f92753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c004aa4a-60f5-428e-8a6e-aece78f92753 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:26.862198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:26.863107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:26.863933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 92/185] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:26.869446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004aa4a-60f5-428e-8a6e-aece78f92753 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:26.933629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa600-0686-4302-8d61-49434badb4fe None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:26.964739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:26.979865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a4e85-9a62-483c-83c8-7bdd1fe1bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:26.980682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4e85-9a62-483c-83c8-7bdd1fe1bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:26.980962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4e85-9a62-483c-83c8-7bdd1fe1bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:26.981234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4e85-9a62-483c-83c8-7bdd1fe1bcb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:26.994767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a4e85-9a62-483c-83c8-7bdd1fe1bcb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-0e0a4e85-9a62-483c-83c8-7bdd1fe1bcb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:27.006208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:27.007166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:27.007987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:27.009177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 94/186] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:27.009459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004aa4a-60f5-428e-8a6e-aece78f92753 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.011432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8eb95-5a9a-4c0d-a59c-c532d40b00ca None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.011913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8eb95-5a9a-4c0d-a59c-c532d40b00ca None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.012185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8eb95-5a9a-4c0d-a59c-c532d40b00ca None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.013488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 95/187] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:27.018166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8eb95-5a9a-4c0d-a59c-c532d40b00ca None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.035232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:27.051079 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93085fd2-cc25-4490-9e31-6f743a7e9d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:27.051789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93085fd2-cc25-4490-9e31-6f743a7e9d14 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.052074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93085fd2-cc25-4490-9e31-6f743a7e9d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.052374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93085fd2-cc25-4490-9e31-6f743a7e9d14 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.053135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93085fd2-cc25-4490-9e31-6f743a7e9d14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:27.053465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93085fd2-cc25-4490-9e31-6f743a7e9d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:27.054979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93085fd2-cc25-4490-9e31-6f743a7e9d14 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:27.064674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 93/188] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:27.070425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b1691-0e86-4914-b404-9c6af1bf2dae None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.070885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b1691-0e86-4914-b404-9c6af1bf2dae None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.071037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b1691-0e86-4914-b404-9c6af1bf2dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.089294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8e6b1691-0e86-4914-b404-9c6af1bf2dae None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:27.090647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8e6b1691-0e86-4914-b404-9c6af1bf2dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:27.124359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8eb95-5a9a-4c0d-a59c-c532d40b00ca None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.158940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:27.159641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.159919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.160190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.160942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:27.161244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:27.162605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:27.165476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 94/189] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:27.166328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b1691-0e86-4914-b404-9c6af1bf2dae None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.183022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 96/190] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:27.184127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.252894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:27.289370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b1691-0e86-4914-b404-9c6af1bf2dae None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.304163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:27.318326 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd24f6c-ace8-4b18-984b-f2b09dd747f3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.326817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328abf-3057-4d6e-8793-1039fd16a142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:27.327726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328abf-3057-4d6e-8793-1039fd16a142 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.328007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328abf-3057-4d6e-8793-1039fd16a142 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.328279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328abf-3057-4d6e-8793-1039fd16a142 None admin] PATH_INFO: `/v3/users/5c59247ab11b4a99b711f9ed0143bf61` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.343439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47328abf-3057-4d6e-8793-1039fd16a142 None admin] RBAC: Authorizing `identity:get_user(user_id=5c59247ab11b4a99b711f9ed0143bf61)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:27.345059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47328abf-3057-4d6e-8793-1039fd16a142 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:27.346771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 95/191] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:27 2022] GET /identity/v3/users/5c59247ab11b4a99b711f9ed0143bf61 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:27.347560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328abf-3057-4d6e-8793-1039fd16a142 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.363037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6715d-25dd-4a54-988e-2dd436e023cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:27.364032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6715d-25dd-4a54-988e-2dd436e023cf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.364319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6715d-25dd-4a54-988e-2dd436e023cf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.364596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6715d-25dd-4a54-988e-2dd436e023cf None admin] PATH_INFO: `/v3/users/5c59247ab11b4a99b711f9ed0143bf61` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.366057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6715d-25dd-4a54-988e-2dd436e023cf None admin] RBAC: Authorizing `identity:get_user(user_id=5c59247ab11b4a99b711f9ed0143bf61)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:27.367465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6715d-25dd-4a54-988e-2dd436e023cf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:27.373818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 97/192] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:27 2022] GET /identity/v3/users/5c59247ab11b4a99b711f9ed0143bf61 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:27.373818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6715d-25dd-4a54-988e-2dd436e023cf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.467444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328abf-3057-4d6e-8793-1039fd16a142 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6715d-25dd-4a54-988e-2dd436e023cf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.497042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:27.499473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22601f71-9ce8-4db2-bf40-d65eaa093522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:27.500331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22601f71-9ce8-4db2-bf40-d65eaa093522 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.500610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22601f71-9ce8-4db2-bf40-d65eaa093522 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.500880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22601f71-9ce8-4db2-bf40-d65eaa093522 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.502054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22601f71-9ce8-4db2-bf40-d65eaa093522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:27.503522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22601f71-9ce8-4db2-bf40-d65eaa093522 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:27.505143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 96/193] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:27.510399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22601f71-9ce8-4db2-bf40-d65eaa093522 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.522217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d726f-e8aa-4af8-adfd-6b2433a28a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:27.522861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d726f-e8aa-4af8-adfd-6b2433a28a6c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.523330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d726f-e8aa-4af8-adfd-6b2433a28a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.523568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d726f-e8aa-4af8-adfd-6b2433a28a6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.550905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:27.599741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d726f-e8aa-4af8-adfd-6b2433a28a6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:27.601390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d726f-e8aa-4af8-adfd-6b2433a28a6c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:27.616766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22601f71-9ce8-4db2-bf40-d65eaa093522 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.652733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7efb-4c7c-46c9-a2b2-abec09b71001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:27.653399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7efb-4c7c-46c9-a2b2-abec09b71001 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.653681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7efb-4c7c-46c9-a2b2-abec09b71001 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.653957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7efb-4c7c-46c9-a2b2-abec09b71001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.707654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 98/194] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:27.708725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d726f-e8aa-4af8-adfd-6b2433a28a6c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.728979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:27.731426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a7efb-4c7c-46c9-a2b2-abec09b71001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:27.733174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a7efb-4c7c-46c9-a2b2-abec09b71001 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:27.775156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:27.821547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d726f-e8aa-4af8-adfd-6b2433a28a6c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.845111 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 97/195] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:27.845767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7efb-4c7c-46c9-a2b2-abec09b71001 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.861941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feb2c2-fbef-44aa-8c19-31a25984b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:27.862629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feb2c2-fbef-44aa-8c19-31a25984b550 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:27.862909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feb2c2-fbef-44aa-8c19-31a25984b550 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:27.863180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feb2c2-fbef-44aa-8c19-31a25984b550 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:27.863941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feb2c2-fbef-44aa-8c19-31a25984b550 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:27.865393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feb2c2-fbef-44aa-8c19-31a25984b550 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:27.886142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 99/196] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:11:27 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:27.893203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feb2c2-fbef-44aa-8c19-31a25984b550 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:27.982656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7efb-4c7c-46c9-a2b2-abec09b71001 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:27.996104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:28.031267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6d45-06fa-43f1-89c3-f437a95c1084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:28.032123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feb2c2-fbef-44aa-8c19-31a25984b550 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:28.032417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6d45-06fa-43f1-89c3-f437a95c1084 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:28.032774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6d45-06fa-43f1-89c3-f437a95c1084 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:28.033123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6d45-06fa-43f1-89c3-f437a95c1084 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:28.034746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6d45-06fa-43f1-89c3-f437a95c1084 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:28.036624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6d45-06fa-43f1-89c3-f437a95c1084 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:28.038796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 98/197] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:27 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:28.044625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6d45-06fa-43f1-89c3-f437a95c1084 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:28.055609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:28.072344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d650f5-4469-4942-8f1b-e7248ef152ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:28.073687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d650f5-4469-4942-8f1b-e7248ef152ae None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:28.074172 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d650f5-4469-4942-8f1b-e7248ef152ae None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:28.074645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d650f5-4469-4942-8f1b-e7248ef152ae None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:28.076269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d650f5-4469-4942-8f1b-e7248ef152ae None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:28.077948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d650f5-4469-4942-8f1b-e7248ef152ae None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:28.079972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 100/198] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:27 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:28.086440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d650f5-4469-4942-8f1b-e7248ef152ae None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:28.182755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6d45-06fa-43f1-89c3-f437a95c1084 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:28.228139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d650f5-4469-4942-8f1b-e7248ef152ae None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:28.232638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc868ba2-5326-4e96-bc8d-ff5ea273fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:28.233666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc868ba2-5326-4e96-bc8d-ff5ea273fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:28.233907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc868ba2-5326-4e96-bc8d-ff5ea273fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:28.234184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc868ba2-5326-4e96-bc8d-ff5ea273fcd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:28.235085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc868ba2-5326-4e96-bc8d-ff5ea273fcd8 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=044d245aa6804c66a019fc432a744109, user.id=5c59247ab11b4a99b711f9ed0143bf61) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:28.235446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc868ba2-5326-4e96-bc8d-ff5ea273fcd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:28.236994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc868ba2-5326-4e96-bc8d-ff5ea273fcd8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:28.249242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 99/199] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:28 2022] GET /identity/v3/role_assignments?user.id=5c59247ab11b4a99b711f9ed0143bf61&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=e07cc5009f624eb9912a837c934e68aa => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:28.273754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f949e9-df61-42db-a524-abe5cc93045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:28.274760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f949e9-df61-42db-a524-abe5cc93045f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:28.275218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f949e9-df61-42db-a524-abe5cc93045f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:28.275739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f949e9-df61-42db-a524-abe5cc93045f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:28.277371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f949e9-df61-42db-a524-abe5cc93045f None admin] RBAC: Adding query filter params (role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=044d245aa6804c66a019fc432a744109, user.id=5c59247ab11b4a99b711f9ed0143bf61) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:28.277875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f949e9-df61-42db-a524-abe5cc93045f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:28.279647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f949e9-df61-42db-a524-abe5cc93045f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:28.287316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 101/200] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:28 2022] GET /identity/v3/role_assignments?user.id=5c59247ab11b4a99b711f9ed0143bf61&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:28.399285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38cd85-4089-412d-abfc-ba82979aed6c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:28.399697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38cd85-4089-412d-abfc-ba82979aed6c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:28.399985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38cd85-4089-412d-abfc-ba82979aed6c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:28.401765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 100/201] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:28.407699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168fc7b-c16c-43e7-8c66-e7b15958f29a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:28.408370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168fc7b-c16c-43e7-8c66-e7b15958f29a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:28.408852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168fc7b-c16c-43e7-8c66-e7b15958f29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:28.414978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:28.430596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e168fc7b-c16c-43e7-8c66-e7b15958f29a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:28.432227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e168fc7b-c16c-43e7-8c66-e7b15958f29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:28.515383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 102/202] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:28.538534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38cd85-4089-412d-abfc-ba82979aed6c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:28.616084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:28.651088 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38cd85-4089-412d-abfc-ba82979aed6c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:28.690590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e762f7-f968-40bb-9d86-19dadde9e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:28.691463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e762f7-f968-40bb-9d86-19dadde9e757 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:28.691796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e762f7-f968-40bb-9d86-19dadde9e757 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:28.692078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e762f7-f968-40bb-9d86-19dadde9e757 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:28.709594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e762f7-f968-40bb-9d86-19dadde9e757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-72e762f7-f968-40bb-9d86-19dadde9e757 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:28.719796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:28.720711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:28.721489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 101/203] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:28.728284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168fc7b-c16c-43e7-8c66-e7b15958f29a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:28.888258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168fc7b-c16c-43e7-8c66-e7b15958f29a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:28.928856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:28.934082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:28.934761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:28.935044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:28.935317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:28.936154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:28.939756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:28.941173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:28.950428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 103/204] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:28.956168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e762f7-f968-40bb-9d86-19dadde9e757 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:29.116285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e762f7-f968-40bb-9d86-19dadde9e757 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:29.156418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:29.162810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ce521-b241-410a-a509-acee292256c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:29.163724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce521-b241-410a-a509-acee292256c3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:29.167912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce521-b241-410a-a509-acee292256c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:29.168260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce521-b241-410a-a509-acee292256c3 None admin] PATH_INFO: `/v3/users/aeb45785ff2545a8b0c9e8c838712dcc` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:29.169465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ce521-b241-410a-a509-acee292256c3 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb45785ff2545a8b0c9e8c838712dcc)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:29.170944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ce521-b241-410a-a509-acee292256c3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:29.172631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 102/205] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:28 2022] GET /identity/v3/users/aeb45785ff2545a8b0c9e8c838712dcc => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:29.178189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:29.291036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03daad-565a-4472-9b4a-fe422219d5b1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:29.330786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceb4b6-cb5c-40e5-9856-c597d7fa4053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:29.331469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceb4b6-cb5c-40e5-9856-c597d7fa4053 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:29.331759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceb4b6-cb5c-40e5-9856-c597d7fa4053 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:29.332021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceb4b6-cb5c-40e5-9856-c597d7fa4053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:29.402576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:29.409795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceb4b6-cb5c-40e5-9856-c597d7fa4053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:29.411208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceb4b6-cb5c-40e5-9856-c597d7fa4053 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:29.514872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 104/206] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:29.524417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ce521-b241-410a-a509-acee292256c3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:29.715818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ce521-b241-410a-a509-acee292256c3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:29.772573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4a018-fbb9-44af-908b-7e4fcdfdd08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:29.773505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a018-fbb9-44af-908b-7e4fcdfdd08c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:29.773814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a018-fbb9-44af-908b-7e4fcdfdd08c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:29.774114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a018-fbb9-44af-908b-7e4fcdfdd08c None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:29.775505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4a018-fbb9-44af-908b-7e4fcdfdd08c None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:29.777145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4a018-fbb9-44af-908b-7e4fcdfdd08c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:29.779491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 103/207] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:29 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:29.785476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceb4b6-cb5c-40e5-9856-c597d7fa4053 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:29.903784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:29.910439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceb4b6-cb5c-40e5-9856-c597d7fa4053 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:29.945796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92d63d-bfe1-4662-9997-fc271ae92c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:29.946454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d63d-bfe1-4662-9997-fc271ae92c02 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:29.946732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d63d-bfe1-4662-9997-fc271ae92c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:29.947002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d63d-bfe1-4662-9997-fc271ae92c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:29.948608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d63d-bfe1-4662-9997-fc271ae92c02 None admin] RBAC: Adding query filter params (role.id=7b21f490e8634fb09bf8b449b44a937a, scope.project.id=7b05c4db805c48ef88067735bdedc47e, user.id=aeb45785ff2545a8b0c9e8c838712dcc) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:29.948809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d63d-bfe1-4662-9997-fc271ae92c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:29.950381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d63d-bfe1-4662-9997-fc271ae92c02 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:29.956743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 105/208] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:29 2022] GET /identity/v3/role_assignments?user.id=aeb45785ff2545a8b0c9e8c838712dcc&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=7b21f490e8634fb09bf8b449b44a937a => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:30.686691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f47f2-9c71-4036-b671-2ae69b540ec6 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:30.687754 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f47f2-9c71-4036-b671-2ae69b540ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:30.688240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f47f2-9c71-4036-b671-2ae69b540ec6 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:30.689845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 104/209] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:30.695141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664223e-dfa1-409a-b10c-ee7feff237ff None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:30.695672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664223e-dfa1-409a-b10c-ee7feff237ff None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:30.696190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664223e-dfa1-409a-b10c-ee7feff237ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:30.718050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7664223e-dfa1-409a-b10c-ee7feff237ff None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:30.719242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7664223e-dfa1-409a-b10c-ee7feff237ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:30.795292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 106/210] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:30.818029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f47f2-9c71-4036-b671-2ae69b540ec6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:30.900569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:30.946789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f47f2-9c71-4036-b671-2ae69b540ec6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:30.964774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29251f-e4de-41e8-92df-2f2da863e649 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:30.965696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29251f-e4de-41e8-92df-2f2da863e649 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:30.966170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29251f-e4de-41e8-92df-2f2da863e649 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:30.968042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 107/211] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:30.974087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3282ef3-28c2-4cd6-905c-f0da144b8b9e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:30.974371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3282ef3-28c2-4cd6-905c-f0da144b8b9e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:30.974646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3282ef3-28c2-4cd6-905c-f0da144b8b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:30.987967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073db8a-0643-4539-9111-df51c3043fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:30.989021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073db8a-0643-4539-9111-df51c3043fae None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:30.989479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073db8a-0643-4539-9111-df51c3043fae None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:30.989942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073db8a-0643-4539-9111-df51c3043fae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:30.993450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b3282ef3-28c2-4cd6-905c-f0da144b8b9e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:30.994865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b3282ef3-28c2-4cd6-905c-f0da144b8b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:31.012241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073db8a-0643-4539-9111-df51c3043fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b073db8a-0643-4539-9111-df51c3043fae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:31.020581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:31.021657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:31.022581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:31.024709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 105/212] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:31.031807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073db8a-0643-4539-9111-df51c3043fae None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.068752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 108/213] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:31.070089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b052ed-92f3-4d50-8b2d-9d76781626ba None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.070376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b052ed-92f3-4d50-8b2d-9d76781626ba None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.070650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b052ed-92f3-4d50-8b2d-9d76781626ba None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.075545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 109/214] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:31.075545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff676a7-d0c0-4161-8163-c84cf297a182 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.075545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff676a7-d0c0-4161-8163-c84cf297a182 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.075545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff676a7-d0c0-4161-8163-c84cf297a182 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.075851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 110/215] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:31.077941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb324a-94bf-492d-b9bc-a4cae5d92e02 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.078233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb324a-94bf-492d-b9bc-a4cae5d92e02 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.078512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb324a-94bf-492d-b9bc-a4cae5d92e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.095029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cccb324a-94bf-492d-b9bc-a4cae5d92e02 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:31.096325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cccb324a-94bf-492d-b9bc-a4cae5d92e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:31.109868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:31.144530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073db8a-0643-4539-9111-df51c3043fae None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.163790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 111/216] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:31.165135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304f7f-070e-4a35-bb4d-768ea12a17a1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.165432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304f7f-070e-4a35-bb4d-768ea12a17a1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.165716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f304f7f-070e-4a35-bb4d-768ea12a17a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.176138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:31.179858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.180516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.180804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.181412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.182254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:31.183210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:31.184669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:31.193624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 106/217] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:31.194527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.195196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8f304f7f-070e-4a35-bb4d-768ea12a17a1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:31.196504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8f304f7f-070e-4a35-bb4d-768ea12a17a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:31.262527 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 112/218] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:31.263382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f304f7f-070e-4a35-bb4d-768ea12a17a1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.297458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea872-3ee0-433c-8c15-ce677ecd3d78 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.330888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db38cc2-1f8f-4c28-bf7b-36950b403b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.331708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38cc2-1f8f-4c28-bf7b-36950b403b5f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.331989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38cc2-1f8f-4c28-bf7b-36950b403b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.332262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38cc2-1f8f-4c28-bf7b-36950b403b5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.333426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db38cc2-1f8f-4c28-bf7b-36950b403b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:31.334890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db38cc2-1f8f-4c28-bf7b-36950b403b5f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:31.336673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 107/219] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:31.337278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db38cc2-1f8f-4c28-bf7b-36950b403b5f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.344210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:31.365639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f304f7f-070e-4a35-bb4d-768ea12a17a1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.398634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f07ab-bd53-48a5-ad6a-d5d7a3a8e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.399423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:31.399535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f07ab-bd53-48a5-ad6a-d5d7a3a8e570 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.399741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f07ab-bd53-48a5-ad6a-d5d7a3a8e570 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.400018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f07ab-bd53-48a5-ad6a-d5d7a3a8e570 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.410612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f07ab-bd53-48a5-ad6a-d5d7a3a8e570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c89f07ab-bd53-48a5-ad6a-d5d7a3a8e570 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:31.418896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:31.419812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:31.420650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 113/220] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:31.421321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f07ab-bd53-48a5-ad6a-d5d7a3a8e570 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.447109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db38cc2-1f8f-4c28-bf7b-36950b403b5f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.481400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f101-8315-47ed-bca5-5f06cfc3c8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.482264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f101-8315-47ed-bca5-5f06cfc3c8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.482539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f101-8315-47ed-bca5-5f06cfc3c8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.482807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f101-8315-47ed-bca5-5f06cfc3c8e1 None admin] PATH_INFO: `/v3/users/df2df47eaee249f190f66244e4a6c844` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.496861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367f101-8315-47ed-bca5-5f06cfc3c8e1 None admin] RBAC: Authorizing `identity:get_user(user_id=df2df47eaee249f190f66244e4a6c844)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:31.498326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367f101-8315-47ed-bca5-5f06cfc3c8e1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:31.500023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 108/221] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/users/df2df47eaee249f190f66244e4a6c844 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:31.504253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f101-8315-47ed-bca5-5f06cfc3c8e1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.519692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f07ab-bd53-48a5-ad6a-d5d7a3a8e570 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.552129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af259e-425c-4195-8ae9-22193e12a2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.552914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af259e-425c-4195-8ae9-22193e12a2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.553192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af259e-425c-4195-8ae9-22193e12a2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.553465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af259e-425c-4195-8ae9-22193e12a2f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.562151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:31.566681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af259e-425c-4195-8ae9-22193e12a2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-89af259e-425c-4195-8ae9-22193e12a2f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:31.575557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:31.576510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:31.577298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 114/222] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:31.577958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af259e-425c-4195-8ae9-22193e12a2f5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.605126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f101-8315-47ed-bca5-5f06cfc3c8e1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.619085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:31.641580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b351b6-220e-4d59-aa6c-2fc612056222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.642245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b351b6-220e-4d59-aa6c-2fc612056222 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.642523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b351b6-220e-4d59-aa6c-2fc612056222 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.642795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b351b6-220e-4d59-aa6c-2fc612056222 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.643533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b351b6-220e-4d59-aa6c-2fc612056222 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:31.644977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b351b6-220e-4d59-aa6c-2fc612056222 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:31.664223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 109/223] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:11:31 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:31.665266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b351b6-220e-4d59-aa6c-2fc612056222 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.692245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af259e-425c-4195-8ae9-22193e12a2f5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.747459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.748154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.748435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.748708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.750063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:31.750375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:31.752254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:31.762006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 115/224] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:31.764315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:31.779789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b351b6-220e-4d59-aa6c-2fc612056222 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.816627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beaaf1-e8c9-4283-98d1-5b4877382f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.817291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beaaf1-e8c9-4283-98d1-5b4877382f49 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.817573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beaaf1-e8c9-4283-98d1-5b4877382f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.817850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beaaf1-e8c9-4283-98d1-5b4877382f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.833646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:31.864814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:31.880619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f41500-fb22-4506-8925-968ac2b5b4a8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:31.907763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beaaf1-e8c9-4283-98d1-5b4877382f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:31.909349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beaaf1-e8c9-4283-98d1-5b4877382f49 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:31.918288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:31.919307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:31.920186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:31.920659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:31.921412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:31.921718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:31.923111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:31.932296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 116/225] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:31.934084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:32.040724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 110/226] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:32.041355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beaaf1-e8c9-4283-98d1-5b4877382f49 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:32.057917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990772a2-2265-43f6-a49f-62c2c8e0acd8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:32.087784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:32.100787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091032b-88c0-4fd8-a35e-f0c380d92c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:32.101673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091032b-88c0-4fd8-a35e-f0c380d92c7d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:32.101953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091032b-88c0-4fd8-a35e-f0c380d92c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:32.102229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091032b-88c0-4fd8-a35e-f0c380d92c7d None admin] PATH_INFO: `/v3/users/5c59247ab11b4a99b711f9ed0143bf61` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:32.103476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091032b-88c0-4fd8-a35e-f0c380d92c7d None admin] RBAC: Authorizing `identity:get_user(user_id=5c59247ab11b4a99b711f9ed0143bf61)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:32.105466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091032b-88c0-4fd8-a35e-f0c380d92c7d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:32.107445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 117/227] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:31 2022] GET /identity/v3/users/5c59247ab11b4a99b711f9ed0143bf61 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:32.108251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091032b-88c0-4fd8-a35e-f0c380d92c7d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:32.142203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:32.187783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beaaf1-e8c9-4283-98d1-5b4877382f49 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:32.220813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091032b-88c0-4fd8-a35e-f0c380d92c7d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:32.238189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb1833-d4c3-4162-8db4-ea8460d5c0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:32.239429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb1833-d4c3-4162-8db4-ea8460d5c0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:32.239759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb1833-d4c3-4162-8db4-ea8460d5c0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:32.240417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb1833-d4c3-4162-8db4-ea8460d5c0e9 None admin] PATH_INFO: `/v3/users/5c59247ab11b4a99b711f9ed0143bf61` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:32.241755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb1833-d4c3-4162-8db4-ea8460d5c0e9 None admin] RBAC: Authorizing `identity:get_user(user_id=5c59247ab11b4a99b711f9ed0143bf61)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:32.243913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb1833-d4c3-4162-8db4-ea8460d5c0e9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:32.247175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 111/228] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:32 2022] GET /identity/v3/users/5c59247ab11b4a99b711f9ed0143bf61 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:32.248174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb1833-d4c3-4162-8db4-ea8460d5c0e9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:32.260399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94232c-06bb-4c55-9e5c-71cc4f2f956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:32.262232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94232c-06bb-4c55-9e5c-71cc4f2f956a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:32.262232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94232c-06bb-4c55-9e5c-71cc4f2f956a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:32.262540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94232c-06bb-4c55-9e5c-71cc4f2f956a None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:32.263849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94232c-06bb-4c55-9e5c-71cc4f2f956a None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:32.265361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94232c-06bb-4c55-9e5c-71cc4f2f956a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:32.267118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 118/229] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:32 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:32.271943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94232c-06bb-4c55-9e5c-71cc4f2f956a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:32.336553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:32.390900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb1833-d4c3-4162-8db4-ea8460d5c0e9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:32.395323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94232c-06bb-4c55-9e5c-71cc4f2f956a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:32.430395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00570f9-30df-4709-888f-a033dcd7f535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:32.431662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00570f9-30df-4709-888f-a033dcd7f535 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:32.432191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00570f9-30df-4709-888f-a033dcd7f535 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:32.433239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00570f9-30df-4709-888f-a033dcd7f535 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:32.434850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00570f9-30df-4709-888f-a033dcd7f535 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:32.437207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00570f9-30df-4709-888f-a033dcd7f535 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:32.438130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d7cc2-e8c5-49b1-a9c9-cdc4cbb72484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:32.438991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d7cc2-e8c5-49b1-a9c9-cdc4cbb72484 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:32.439264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d7cc2-e8c5-49b1-a9c9-cdc4cbb72484 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:32.439576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d7cc2-e8c5-49b1-a9c9-cdc4cbb72484 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:32.440621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 112/230] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:32 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:32.440825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d7cc2-e8c5-49b1-a9c9-cdc4cbb72484 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:32.441812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00570f9-30df-4709-888f-a033dcd7f535 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:32.442347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d7cc2-e8c5-49b1-a9c9-cdc4cbb72484 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:32.444246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 119/231] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:32 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:32.450498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d7cc2-e8c5-49b1-a9c9-cdc4cbb72484 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:32.475486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:32.560531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d7cc2-e8c5-49b1-a9c9-cdc4cbb72484 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:32.584411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00570f9-30df-4709-888f-a033dcd7f535 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:32.595405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:32.605983 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a79ca6-e55d-4d97-b054-3b3aa09e6d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:32.606989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a79ca6-e55d-4d97-b054-3b3aa09e6d92 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:32.607275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a79ca6-e55d-4d97-b054-3b3aa09e6d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:32.607589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a79ca6-e55d-4d97-b054-3b3aa09e6d92 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109/users/5c59247ab11b4a99b711f9ed0143bf61/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:32.609828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a79ca6-e55d-4d97-b054-3b3aa09e6d92 None admin] RBAC: Authorizing `identity:create_grant(project_id=044d245aa6804c66a019fc432a744109, user_id=5c59247ab11b4a99b711f9ed0143bf61, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:32.611292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a79ca6-e55d-4d97-b054-3b3aa09e6d92 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:32.631724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 120/232] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:32 2022] PUT /identity/v3/projects/044d245aa6804c66a019fc432a744109/users/5c59247ab11b4a99b711f9ed0143bf61/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:32.632780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a79ca6-e55d-4d97-b054-3b3aa09e6d92 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:32.634847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756ade0-d348-4da4-b04c-d4060c9de455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:32.635931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756ade0-d348-4da4-b04c-d4060c9de455 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:32.636425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756ade0-d348-4da4-b04c-d4060c9de455 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:32.636968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756ade0-d348-4da4-b04c-d4060c9de455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:32.638038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756ade0-d348-4da4-b04c-d4060c9de455 None admin] RBAC: Adding query filter params (role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=044d245aa6804c66a019fc432a744109, user.id=df2df47eaee249f190f66244e4a6c844) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:32.638592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756ade0-d348-4da4-b04c-d4060c9de455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:32.640465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756ade0-d348-4da4-b04c-d4060c9de455 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:32.647116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 113/233] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:32 2022] GET /identity/v3/role_assignments?user.id=df2df47eaee249f190f66244e4a6c844&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:32.826718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a79ca6-e55d-4d97-b054-3b3aa09e6d92 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:32.876618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93128f-30d9-43d1-8250-3ecaa12c7232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:32.877458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93128f-30d9-43d1-8250-3ecaa12c7232 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:32.877740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93128f-30d9-43d1-8250-3ecaa12c7232 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:32.878020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93128f-30d9-43d1-8250-3ecaa12c7232 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109/users/5c59247ab11b4a99b711f9ed0143bf61/roles/64cbdf4e69fa400b8b2f8ed13ebe5367` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:32.886087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93128f-30d9-43d1-8250-3ecaa12c7232 None admin] RBAC: Authorizing `identity:create_grant(project_id=044d245aa6804c66a019fc432a744109, user_id=5c59247ab11b4a99b711f9ed0143bf61, role_id=64cbdf4e69fa400b8b2f8ed13ebe5367)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:32.887669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93128f-30d9-43d1-8250-3ecaa12c7232 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:32.909649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 121/234] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:32 2022] PUT /identity/v3/projects/044d245aa6804c66a019fc432a744109/users/5c59247ab11b4a99b711f9ed0143bf61/roles/64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:34.339370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18c145-d1ca-4be1-8015-245d761f1352 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:34.339886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18c145-d1ca-4be1-8015-245d761f1352 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:34.339990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18c145-d1ca-4be1-8015-245d761f1352 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:34.341758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 114/235] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:34.346760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc89d-98b9-42bd-b0c2-04d9dcc198c4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:34.347153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc89d-98b9-42bd-b0c2-04d9dcc198c4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:34.347314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc89d-98b9-42bd-b0c2-04d9dcc198c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:34.368530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bbdcc89d-98b9-42bd-b0c2-04d9dcc198c4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:34.369730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bbdcc89d-98b9-42bd-b0c2-04d9dcc198c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:34.412322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4874e9-1508-4558-87eb-3da9c8fee11d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:34.412624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4874e9-1508-4558-87eb-3da9c8fee11d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:34.412883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4874e9-1508-4558-87eb-3da9c8fee11d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:34.414265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 115/236] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:11:34.419920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba50d1b-5e08-43ad-bc7a-665dc1613ad7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:34.420207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba50d1b-5e08-43ad-bc7a-665dc1613ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:34.420489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba50d1b-5e08-43ad-bc7a-665dc1613ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:34.433777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:34.440003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dba50d1b-5e08-43ad-bc7a-665dc1613ad7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:34.441200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dba50d1b-5e08-43ad-bc7a-665dc1613ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:34.481911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 122/237] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:34.508511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcc89d-98b9-42bd-b0c2-04d9dcc198c4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:34.584057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:34.585618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 116/238] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:34.615472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba50d1b-5e08-43ad-bc7a-665dc1613ad7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:34.701440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcc89d-98b9-42bd-b0c2-04d9dcc198c4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:34.772979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f38ea-3d62-40bb-8835-0c889c2d8896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:34.774156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f38ea-3d62-40bb-8835-0c889c2d8896 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:34.774653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f38ea-3d62-40bb-8835-0c889c2d8896 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:34.775114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f38ea-3d62-40bb-8835-0c889c2d8896 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:34.778327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f38ea-3d62-40bb-8835-0c889c2d8896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:34.780162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f38ea-3d62-40bb-8835-0c889c2d8896 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:34.782110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba50d1b-5e08-43ad-bc7a-665dc1613ad7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:34.786308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 123/239] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:34.792966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f38ea-3d62-40bb-8835-0c889c2d8896 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:34.850712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fc070-1266-4a6a-b868-b638f32ee652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:34.851951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fc070-1266-4a6a-b868-b638f32ee652 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:34.852458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fc070-1266-4a6a-b868-b638f32ee652 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:34.852940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fc070-1266-4a6a-b868-b638f32ee652 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:34.873940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fc070-1266-4a6a-b868-b638f32ee652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-dd1fc070-1266-4a6a-b868-b638f32ee652 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:34.883353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:34.884250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:34.885031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:34.886579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:34.888967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 117/240] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:34.895597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fc070-1266-4a6a-b868-b638f32ee652 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:34.994221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:35.000332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f38ea-3d62-40bb-8835-0c889c2d8896 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:35.061880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703e1d-6fea-4353-80c3-9cbc565a5b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:35.062566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703e1d-6fea-4353-80c3-9cbc565a5b71 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.062845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703e1d-6fea-4353-80c3-9cbc565a5b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.063116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703e1d-6fea-4353-80c3-9cbc565a5b71 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.063869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d703e1d-6fea-4353-80c3-9cbc565a5b71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:35.065313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d703e1d-6fea-4353-80c3-9cbc565a5b71 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:35.078008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fc070-1266-4a6a-b868-b638f32ee652 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:35.102840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 124/241] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:11:34 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:35.130363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:35.130975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.131258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.131692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.132462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:35.132778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:35.134254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:35.144843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 118/242] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:35.150777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703e1d-6fea-4353-80c3-9cbc565a5b71 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:35.264383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:35.268449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703e1d-6fea-4353-80c3-9cbc565a5b71 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:35.306965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50617266-8a07-4a87-a89e-503fd8606ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:35.308066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617266-8a07-4a87-a89e-503fd8606ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.308414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617266-8a07-4a87-a89e-503fd8606ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.308695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617266-8a07-4a87-a89e-503fd8606ca2 None admin] PATH_INFO: `/v3/users/72e821ca8139466cbc5fcd065effa45c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.326912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50617266-8a07-4a87-a89e-503fd8606ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=72e821ca8139466cbc5fcd065effa45c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:35.328422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50617266-8a07-4a87-a89e-503fd8606ca2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:35.330529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 125/243] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:35 2022] GET /identity/v3/users/72e821ca8139466cbc5fcd065effa45c => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:35.336092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:35.416155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986fdb6-d385-483c-b7ea-5d2b297779f4 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.416672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986fdb6-d385-483c-b7ea-5d2b297779f4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.416672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986fdb6-d385-483c-b7ea-5d2b297779f4 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.418048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 126/244] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:11:35.423504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f365e12-8720-41e9-a8de-4020d0e1c9e1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.423795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f365e12-8720-41e9-a8de-4020d0e1c9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.424077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f365e12-8720-41e9-a8de-4020d0e1c9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.443856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4f365e12-8720-41e9-a8de-4020d0e1c9e1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:35.445386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4f365e12-8720-41e9-a8de-4020d0e1c9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:35.468905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24df1a-b599-4511-a217-2eb895f326b1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:35.501252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:35.518588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb2469-7e6f-4adf-8f1e-b26ea12f475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:35.519702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb2469-7e6f-4adf-8f1e-b26ea12f475d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.520227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb2469-7e6f-4adf-8f1e-b26ea12f475d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.521226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb2469-7e6f-4adf-8f1e-b26ea12f475d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.540296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 127/245] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:35.565294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f365e12-8720-41e9-a8de-4020d0e1c9e1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:35.616485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb2469-7e6f-4adf-8f1e-b26ea12f475d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:35.618005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb2469-7e6f-4adf-8f1e-b26ea12f475d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:35.692292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:35.709865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f365e12-8720-41e9-a8de-4020d0e1c9e1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:35.761689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b00042-9e8d-43e8-ab81-4a39a04fc3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:35.762501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b00042-9e8d-43e8-ab81-4a39a04fc3ed None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.762783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b00042-9e8d-43e8-ab81-4a39a04fc3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.763055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b00042-9e8d-43e8-ab81-4a39a04fc3ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.768949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 119/246] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:35.770864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47de67-b9e0-4506-bff4-78ef9d49a404 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.771194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47de67-b9e0-4506-bff4-78ef9d49a404 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.771929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47de67-b9e0-4506-bff4-78ef9d49a404 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.773809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 120/247] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:35.774442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b00042-9e8d-43e8-ab81-4a39a04fc3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:35.779836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47de67-b9e0-4506-bff4-78ef9d49a404 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-23b00042-9e8d-43e8-ab81-4a39a04fc3ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:35.784760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:35.785688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:35.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 128/248] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:11:35.787720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472f5cb-bc06-4368-9293-78e030d68c8d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.787819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472f5cb-bc06-4368-9293-78e030d68c8d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.788947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472f5cb-bc06-4368-9293-78e030d68c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.807992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0472f5cb-bc06-4368-9293-78e030d68c8d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:35.810301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0472f5cb-bc06-4368-9293-78e030d68c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:35.812510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:35.896153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 129/249] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:35.897416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472f5cb-bc06-4368-9293-78e030d68c8d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:35.917646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47de67-b9e0-4506-bff4-78ef9d49a404 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:35.961914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19710e-894a-4d89-b8d6-5daa79bafe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:35.962932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19710e-894a-4d89-b8d6-5daa79bafe3c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.963218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19710e-894a-4d89-b8d6-5daa79bafe3c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.963524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19710e-894a-4d89-b8d6-5daa79bafe3c None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.965237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19710e-894a-4d89-b8d6-5daa79bafe3c None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:35.966692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19710e-894a-4d89-b8d6-5daa79bafe3c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:35.969775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 121/250] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:35 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:35.971056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad7199-ea1d-422c-9830-9fdd01c0b8f7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:35.971360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad7199-ea1d-422c-9830-9fdd01c0b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:35.971638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad7199-ea1d-422c-9830-9fdd01c0b8f7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:35.975422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 122/251] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:35.976235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad7199-ea1d-422c-9830-9fdd01c0b8f7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:36.004645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:36.029478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472f5cb-bc06-4368-9293-78e030d68c8d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:36.065275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:36.065949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:36.066232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:36.066505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:36.067310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:36.067644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:36.069068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:36.077604 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:36.078452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 130/252] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:36.079208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:36.095979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad7199-ea1d-422c-9830-9fdd01c0b8f7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:36.142699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d17f0f-fa82-4f2c-9d95-72216b24cfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:36.143545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17f0f-fa82-4f2c-9d95-72216b24cfef None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:36.143823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17f0f-fa82-4f2c-9d95-72216b24cfef None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:36.144349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17f0f-fa82-4f2c-9d95-72216b24cfef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:36.159499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d17f0f-fa82-4f2c-9d95-72216b24cfef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c3d17f0f-fa82-4f2c-9d95-72216b24cfef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:36.170697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:36.171618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:36.172416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 123/253] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:36.173785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b708c8-87db-4abd-9795-af06bf372f08 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:36.173785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b708c8-87db-4abd-9795-af06bf372f08 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:36.173785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b708c8-87db-4abd-9795-af06bf372f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:36.192577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e3b708c8-87db-4abd-9795-af06bf372f08 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:36.193898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e3b708c8-87db-4abd-9795-af06bf372f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:36.216540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a514f1-53e7-40ba-8b79-69161cafeee8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:36.251820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:36.252500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:36.252760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:36.253034 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:36.253876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] RBAC: Adding query filter params (role.id=7b21f490e8634fb09bf8b449b44a937a, scope.project.id=044d245aa6804c66a019fc432a744109, user.id=72e821ca8139466cbc5fcd065effa45c) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:36.254187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:36.255643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:36.261416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 131/254] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:36 2022] GET /identity/v3/role_assignments?user.id=72e821ca8139466cbc5fcd065effa45c&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=7b21f490e8634fb09bf8b449b44a937a => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:36.262145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:36.265993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:36.275498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 124/255] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:36.277133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b708c8-87db-4abd-9795-af06bf372f08 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:36.448851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:36.481783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547ba6-64f1-4986-8143-8404fafef82e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:36.483438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b708c8-87db-4abd-9795-af06bf372f08 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:36.530730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:36.532898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:36.534545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:36.534966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:36.535889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:36.538251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:36.539859 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:36.546248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee373a2-b761-4f35-8f5a-6773bdaab8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:36.548226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee373a2-b761-4f35-8f5a-6773bdaab8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:36.550940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee373a2-b761-4f35-8f5a-6773bdaab8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:36.551416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee373a2-b761-4f35-8f5a-6773bdaab8b7 None admin] PATH_INFO: `/v3/users/df2df47eaee249f190f66244e4a6c844` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:36.552870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee373a2-b761-4f35-8f5a-6773bdaab8b7 None admin] RBAC: Authorizing `identity:get_user(user_id=df2df47eaee249f190f66244e4a6c844)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:36.555061 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee373a2-b761-4f35-8f5a-6773bdaab8b7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:36.559883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 132/256] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:36 2022] GET /identity/v3/users/df2df47eaee249f190f66244e4a6c844 => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:36.561662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 125/257] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:36.562705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:36.568596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee373a2-b761-4f35-8f5a-6773bdaab8b7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:36.647877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:36.713253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee373a2-b761-4f35-8f5a-6773bdaab8b7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:36.757358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b13d8-90b6-420f-bc9c-3a466e1aaf1e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:36.803269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c798b8-32fc-4e97-9983-708a121d1c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:36.804163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c798b8-32fc-4e97-9983-708a121d1c54 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:36.804439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c798b8-32fc-4e97-9983-708a121d1c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:36.804707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c798b8-32fc-4e97-9983-708a121d1c54 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:36.809821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c798b8-32fc-4e97-9983-708a121d1c54 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:36.813188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c798b8-32fc-4e97-9983-708a121d1c54 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:36.816472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 133/258] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:36 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:36.817557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c798b8-32fc-4e97-9983-708a121d1c54 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:36.832366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824946a-15bc-4456-a640-1bd0c732a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:36.833187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824946a-15bc-4456-a640-1bd0c732a8de None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:36.833463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824946a-15bc-4456-a640-1bd0c732a8de None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:36.833758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824946a-15bc-4456-a640-1bd0c732a8de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:36.858727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824946a-15bc-4456-a640-1bd0c732a8de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3824946a-15bc-4456-a640-1bd0c732a8de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:36.874112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:36.875122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:36.875960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 126/259] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:36.876736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824946a-15bc-4456-a640-1bd0c732a8de None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:36.893438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:37.004058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c798b8-32fc-4e97-9983-708a121d1c54 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:37.069411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2654b9-2848-4d09-9205-81c49190f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:37.070205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2654b9-2848-4d09-9205-81c49190f393 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:37.070481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2654b9-2848-4d09-9205-81c49190f393 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:37.070752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2654b9-2848-4d09-9205-81c49190f393 None admin] PATH_INFO: `/v3/users/5c59247ab11b4a99b711f9ed0143bf61` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:37.075416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2654b9-2848-4d09-9205-81c49190f393 None admin] RBAC: Authorizing `identity:get_user(user_id=5c59247ab11b4a99b711f9ed0143bf61)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:37.076859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2654b9-2848-4d09-9205-81c49190f393 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:37.078977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 134/260] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:36 2022] GET /identity/v3/users/5c59247ab11b4a99b711f9ed0143bf61 => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:37.084184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2654b9-2848-4d09-9205-81c49190f393 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:37.103693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:37.105477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824946a-15bc-4456-a640-1bd0c732a8de None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:37.175826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bb3e1-4726-4e6c-8f6f-b8c3dbec6081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:37.176722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb3e1-4726-4e6c-8f6f-b8c3dbec6081 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:37.176995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb3e1-4726-4e6c-8f6f-b8c3dbec6081 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:37.177259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb3e1-4726-4e6c-8f6f-b8c3dbec6081 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109/users/df2df47eaee249f190f66244e4a6c844/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:37.181058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bb3e1-4726-4e6c-8f6f-b8c3dbec6081 None admin] RBAC: Authorizing `identity:create_grant(project_id=044d245aa6804c66a019fc432a744109, user_id=df2df47eaee249f190f66244e4a6c844, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:37.184669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bb3e1-4726-4e6c-8f6f-b8c3dbec6081 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:37.204290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 127/261] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:36 2022] PUT /identity/v3/projects/044d245aa6804c66a019fc432a744109/users/df2df47eaee249f190f66244e4a6c844/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:37.206218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bb3e1-4726-4e6c-8f6f-b8c3dbec6081 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:37.292750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2654b9-2848-4d09-9205-81c49190f393 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:37.316691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:37.379008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:37.379640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:37.379921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:37.380196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:37.380960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:37.381271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:37.382652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:37.401092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 135/262] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:37.407564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:37.410314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bb3e1-4726-4e6c-8f6f-b8c3dbec6081 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:37.524120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf50f0-d480-4cd8-a7dd-20685b738e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:37.524912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf50f0-d480-4cd8-a7dd-20685b738e58 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:37.525077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf50f0-d480-4cd8-a7dd-20685b738e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:37.525352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf50f0-d480-4cd8-a7dd-20685b738e58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:37.568323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:37.628993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f329fb1-87b1-4035-a3c5-b2c7edb108f2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:37.663765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf50f0-d480-4cd8-a7dd-20685b738e58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:37.665293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf50f0-d480-4cd8-a7dd-20685b738e58 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:37.716160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c2f53-8be2-45e0-9481-baeccca62d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:37.718267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c2f53-8be2-45e0-9481-baeccca62d39 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:37.718267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c2f53-8be2-45e0-9481-baeccca62d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:37.718267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c2f53-8be2-45e0-9481-baeccca62d39 None admin] PATH_INFO: `/v3/users/5c59247ab11b4a99b711f9ed0143bf61` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:37.728913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c2f53-8be2-45e0-9481-baeccca62d39 None admin] RBAC: Authorizing `identity:get_user(user_id=5c59247ab11b4a99b711f9ed0143bf61)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:37.728913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c2f53-8be2-45e0-9481-baeccca62d39 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:37.728913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 136/263] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:37 2022] GET /identity/v3/users/5c59247ab11b4a99b711f9ed0143bf61 => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:37.733499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c2f53-8be2-45e0-9481-baeccca62d39 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:37.734164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:37.779919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 128/264] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:37.785428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf50f0-d480-4cd8-a7dd-20685b738e58 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:37.918199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf50f0-d480-4cd8-a7dd-20685b738e58 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:37.943222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c2f53-8be2-45e0-9481-baeccca62d39 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:37.955206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9408b-825d-4ca3-8b08-5c45cc2864d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:37.956150 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9408b-825d-4ca3-8b08-5c45cc2864d6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:37.956433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9408b-825d-4ca3-8b08-5c45cc2864d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:37.956711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9408b-825d-4ca3-8b08-5c45cc2864d6 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:37.958052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9408b-825d-4ca3-8b08-5c45cc2864d6 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:37.959586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9408b-825d-4ca3-8b08-5c45cc2864d6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:37.961700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 129/265] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:37 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:37.989511 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9408b-825d-4ca3-8b08-5c45cc2864d6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:38.003793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c719b-28f3-4a54-8987-3319c5adce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:38.004457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c719b-28f3-4a54-8987-3319c5adce73 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.004735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c719b-28f3-4a54-8987-3319c5adce73 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.005008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c719b-28f3-4a54-8987-3319c5adce73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.018282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:38.051779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:38.133915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c719b-28f3-4a54-8987-3319c5adce73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:38.136525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c719b-28f3-4a54-8987-3319c5adce73 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:38.171449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9408b-825d-4ca3-8b08-5c45cc2864d6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:38.201413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b105160-4673-46dc-9617-ce697dd1c662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:38.202061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105160-4673-46dc-9617-ce697dd1c662 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.202345 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105160-4673-46dc-9617-ce697dd1c662 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.202613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105160-4673-46dc-9617-ce697dd1c662 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.203478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b105160-4673-46dc-9617-ce697dd1c662 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=044d245aa6804c66a019fc432a744109, user.id=5c59247ab11b4a99b711f9ed0143bf61) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:38.203790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b105160-4673-46dc-9617-ce697dd1c662 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:38.205238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b105160-4673-46dc-9617-ce697dd1c662 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:38.211370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 130/266] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:37 2022] GET /identity/v3/role_assignments?user.id=5c59247ab11b4a99b711f9ed0143bf61&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=e07cc5009f624eb9912a837c934e68aa => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:38.216870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b713e-09f6-4adb-9f39-4b2369f47659 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.217143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b713e-09f6-4adb-9f39-4b2369f47659 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.217419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b713e-09f6-4adb-9f39-4b2369f47659 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.218915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 131/267] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:38.228880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c9e1-4c33-47b2-836b-f28d7cd0c0da None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.229717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c9e1-4c33-47b2-836b-f28d7cd0c0da None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.230050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c9e1-4c33-47b2-836b-f28d7cd0c0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.259043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9266c9e1-4c33-47b2-836b-f28d7cd0c0da None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:38.260267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9266c9e1-4c33-47b2-836b-f28d7cd0c0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:38.272908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 137/268] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 17:11:38.284628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c719b-28f3-4a54-8987-3319c5adce73 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:38.378624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:38.379591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 132/269] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:38.424143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266c9e1-4c33-47b2-836b-f28d7cd0c0da None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:38.465305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:38.509244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c719b-28f3-4a54-8987-3319c5adce73 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:38.555983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266c9e1-4c33-47b2-836b-f28d7cd0c0da None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:38.590971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312592b-5f7b-41ab-9f17-ffd328660e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:38.591884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312592b-5f7b-41ab-9f17-ffd328660e3b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.592169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312592b-5f7b-41ab-9f17-ffd328660e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.592694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312592b-5f7b-41ab-9f17-ffd328660e3b None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.593661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f46be2-0bc8-422f-8992-96f8220e006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:38.593884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312592b-5f7b-41ab-9f17-ffd328660e3b None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:38.594459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46be2-0bc8-422f-8992-96f8220e006d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.594738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46be2-0bc8-422f-8992-96f8220e006d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.595008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46be2-0bc8-422f-8992-96f8220e006d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.595364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312592b-5f7b-41ab-9f17-ffd328660e3b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:38.596218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f46be2-0bc8-422f-8992-96f8220e006d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:38.597667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f46be2-0bc8-422f-8992-96f8220e006d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:38.599544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 133/271] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:38.600486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 138/271] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:38 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:38.600817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bb5e-039c-41fe-ab25-297fe3abedf1 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.601101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bb5e-039c-41fe-ab25-297fe3abedf1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.601371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bb5e-039c-41fe-ab25-297fe3abedf1 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.602898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 134/272] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:38.603273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312592b-5f7b-41ab-9f17-ffd328660e3b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:38.606259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932bb5e-039c-41fe-ab25-297fe3abedf1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:38.725864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:38.732237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312592b-5f7b-41ab-9f17-ffd328660e3b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:38.740982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932bb5e-039c-41fe-ab25-297fe3abedf1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:38.752647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:38.770973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159f282-d61a-4502-815c-26b1d5430b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:38.771985 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159f282-d61a-4502-815c-26b1d5430b4a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.772719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159f282-d61a-4502-815c-26b1d5430b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.773169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159f282-d61a-4502-815c-26b1d5430b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.774169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159f282-d61a-4502-815c-26b1d5430b4a None admin] RBAC: Adding query filter params (role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=044d245aa6804c66a019fc432a744109, user.id=5c59247ab11b4a99b711f9ed0143bf61) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:38.774653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159f282-d61a-4502-815c-26b1d5430b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:38.777508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159f282-d61a-4502-815c-26b1d5430b4a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:38.786694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68927-9b20-410c-91bb-bd73a4fcd87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:38.787194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 139/273] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:38 2022] GET /identity/v3/role_assignments?user.id=5c59247ab11b4a99b711f9ed0143bf61&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:38.788763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be322798-5392-4a8d-9666-3b88d7758c52 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.789267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be322798-5392-4a8d-9666-3b88d7758c52 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.789733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be322798-5392-4a8d-9666-3b88d7758c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.792229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68927-9b20-410c-91bb-bd73a4fcd87c None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:38.792550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68927-9b20-410c-91bb-bd73a4fcd87c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:38.792833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68927-9b20-410c-91bb-bd73a4fcd87c None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:38.793672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d68927-9b20-410c-91bb-bd73a4fcd87c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:38.795164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d68927-9b20-410c-91bb-bd73a4fcd87c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:38.817374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-be322798-5392-4a8d-9666-3b88d7758c52 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:38.820932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-be322798-5392-4a8d-9666-3b88d7758c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:38.835711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 135/274] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:11:38 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:11:38.911786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 140/275] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:38.938713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68927-9b20-410c-91bb-bd73a4fcd87c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:39.081899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68927-9b20-410c-91bb-bd73a4fcd87c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:39.126294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3798b-d3fd-4bca-b221-fa52fb0e601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:39.127112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3798b-d3fd-4bca-b221-fa52fb0e601f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:39.127448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3798b-d3fd-4bca-b221-fa52fb0e601f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:39.127726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3798b-d3fd-4bca-b221-fa52fb0e601f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:39.145692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3798b-d3fd-4bca-b221-fa52fb0e601f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cdc3798b-d3fd-4bca-b221-fa52fb0e601f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:39.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:39.154704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:39.155594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:39.159407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 136/276] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:39.161110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be322798-5392-4a8d-9666-3b88d7758c52 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:39.217469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c976ef-a3bf-45b1-aafd-5af0c6f63e8c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:39.217797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c976ef-a3bf-45b1-aafd-5af0c6f63e8c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:39.218026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c976ef-a3bf-45b1-aafd-5af0c6f63e8c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:39.219717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 137/277] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:39.225527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c037c49-8759-4e12-a015-0a46645de860 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:39.225813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c037c49-8759-4e12-a015-0a46645de860 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:39.226090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c037c49-8759-4e12-a015-0a46645de860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:39.246065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5c037c49-8759-4e12-a015-0a46645de860 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:39.247460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5c037c49-8759-4e12-a015-0a46645de860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:39.250240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:39.267786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:39.271759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be322798-5392-4a8d-9666-3b88d7758c52 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:39.308344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:39.309236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:39.309344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:39.309557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:39.310308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:39.310618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:39.312032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:39.321874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 141/278] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:39.327857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:39.340899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 138/279] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:39.367420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c037c49-8759-4e12-a015-0a46645de860 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:39.459748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99e2ea-400b-4b17-94f2-de5bcbaad503 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:39.496095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c037c49-8759-4e12-a015-0a46645de860 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:39.520089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae026230-6ce8-454b-bda5-d5195050d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:39.521057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae026230-6ce8-454b-bda5-d5195050d2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:39.521367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae026230-6ce8-454b-bda5-d5195050d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:39.521671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae026230-6ce8-454b-bda5-d5195050d2c5 None admin] PATH_INFO: `/v3/users/0807822c7b3d4c92b81c4fa42f387832` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:39.543964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0c9b-f413-436b-8650-6237f77c12d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:39.545700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0c9b-f413-436b-8650-6237f77c12d1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:39.546186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0c9b-f413-436b-8650-6237f77c12d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:39.546887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0c9b-f413-436b-8650-6237f77c12d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:39.556275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:39.563030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0c9b-f413-436b-8650-6237f77c12d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:39.564130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae026230-6ce8-454b-bda5-d5195050d2c5 None admin] RBAC: Authorizing `identity:get_user(user_id=0807822c7b3d4c92b81c4fa42f387832)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:39.565782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae026230-6ce8-454b-bda5-d5195050d2c5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:39.568189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 142/280] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:39 2022] GET /identity/v3/users/0807822c7b3d4c92b81c4fa42f387832 => generated 293 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-1c8c0c9b-f413-436b-8650-6237f77c12d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:39.572514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:39.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:39.574755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:39.575387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae026230-6ce8-454b-bda5-d5195050d2c5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:39.577599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 139/281] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:39.588673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0c9b-f413-436b-8650-6237f77c12d1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:39.603098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:39.698376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae026230-6ce8-454b-bda5-d5195050d2c5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:39.737522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:39.738762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:39.739252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:39.739750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:39.740894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] RBAC: Adding query filter params (role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, user.id=0807822c7b3d4c92b81c4fa42f387832, scope.system=all) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:39.741424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:39.743132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:39.751040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 143/282] 10.222.0.33 () {56 vars in 1308 bytes} [Sat May 28 17:11:39 2022] GET /identity/v3/role_assignments?user.id=0807822c7b3d4c92b81c4fa42f387832&scope.system=all&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:39.804785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0c9b-f413-436b-8650-6237f77c12d1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:39.845219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:39.845911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:39.846192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:39.846461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:39.847229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:39.847564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:39.848979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:39.859818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 140/283] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:39.865389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:40.002181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:40.036316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a7317-d6a4-48b8-a484-4a36139b1f09 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:40.085157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2863-6536-4827-a3ab-d7379f3f9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:40.086134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2863-6536-4827-a3ab-d7379f3f9240 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:40.086447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2863-6536-4827-a3ab-d7379f3f9240 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:40.086833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2863-6536-4827-a3ab-d7379f3f9240 None admin] PATH_INFO: `/v3/users/72e821ca8139466cbc5fcd065effa45c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:40.088976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2863-6536-4827-a3ab-d7379f3f9240 None admin] RBAC: Authorizing `identity:get_user(user_id=72e821ca8139466cbc5fcd065effa45c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:40.091047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2863-6536-4827-a3ab-d7379f3f9240 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:40.094049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 144/284] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:39 2022] GET /identity/v3/users/72e821ca8139466cbc5fcd065effa45c => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:40.101581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:40.220681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:40.266271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea3dc7-770c-4efa-9258-679b52a0bc09 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:40.328478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd8e27-6a2f-4257-ace4-6a8fbc8e368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:40.331122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd8e27-6a2f-4257-ace4-6a8fbc8e368c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:40.331437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd8e27-6a2f-4257-ace4-6a8fbc8e368c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:40.331708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd8e27-6a2f-4257-ace4-6a8fbc8e368c None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:40.333898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd8e27-6a2f-4257-ace4-6a8fbc8e368c None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:40.341144 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd8e27-6a2f-4257-ace4-6a8fbc8e368c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:40.343229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 141/285] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:40 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:40.348952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2863-6536-4827-a3ab-d7379f3f9240 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:40.377462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3a25f-cf52-4f8a-ba10-3d7ce0b97b95 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:40.377769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3a25f-cf52-4f8a-ba10-3d7ce0b97b95 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:40.379399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3a25f-cf52-4f8a-ba10-3d7ce0b97b95 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:40.379791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 142/286] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:40.387738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46307031-c2ee-4856-a9a2-b3d2404c2dc9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:40.388026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46307031-c2ee-4856-a9a2-b3d2404c2dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:40.388292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46307031-c2ee-4856-a9a2-b3d2404c2dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:40.422666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-46307031-c2ee-4856-a9a2-b3d2404c2dc9 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:40.424682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-46307031-c2ee-4856-a9a2-b3d2404c2dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:40.538039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 143/287] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:40.595418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46307031-c2ee-4856-a9a2-b3d2404c2dc9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:40.596566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2863-6536-4827-a3ab-d7379f3f9240 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:40.655251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510be7e-9931-4396-bb69-694e755e3c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:40.680003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510be7e-9931-4396-bb69-694e755e3c54 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:40.680003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510be7e-9931-4396-bb69-694e755e3c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:40.680003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510be7e-9931-4396-bb69-694e755e3c54 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109/users/72e821ca8139466cbc5fcd065effa45c/roles/7b21f490e8634fb09bf8b449b44a937a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:40.680003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510be7e-9931-4396-bb69-694e755e3c54 None admin] RBAC: Authorizing `identity:create_grant(project_id=044d245aa6804c66a019fc432a744109, user_id=72e821ca8139466cbc5fcd065effa45c, role_id=7b21f490e8634fb09bf8b449b44a937a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:40.680003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510be7e-9931-4396-bb69-694e755e3c54 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:40.689094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 145/288] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:11:40 2022] PUT /identity/v3/projects/044d245aa6804c66a019fc432a744109/users/72e821ca8139466cbc5fcd065effa45c/roles/7b21f490e8634fb09bf8b449b44a937a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:40.735941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:40.828362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46307031-c2ee-4856-a9a2-b3d2404c2dc9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:40.877241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9843cae-9261-404a-b7e6-8da21763f339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:40.878065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9843cae-9261-404a-b7e6-8da21763f339 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:40.878343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9843cae-9261-404a-b7e6-8da21763f339 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:40.878615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9843cae-9261-404a-b7e6-8da21763f339 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:40.895473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9843cae-9261-404a-b7e6-8da21763f339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b9843cae-9261-404a-b7e6-8da21763f339 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:40.903745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:40.905910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:40.907585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 144/289] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:40.914628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510be7e-9931-4396-bb69-694e755e3c54 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:40.935182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:41.083835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510be7e-9931-4396-bb69-694e755e3c54 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:41.127058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:41.127768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:41.128051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:41.128334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:41.129088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:41.129397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:41.130776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:41.144076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 146/290] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:41.148418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9843cae-9261-404a-b7e6-8da21763f339 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:41.302804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9843cae-9261-404a-b7e6-8da21763f339 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:41.325534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:41.350984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb66fe9-7dff-4d43-aa93-4c94deef063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:41.351830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb66fe9-7dff-4d43-aa93-4c94deef063f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:41.352113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb66fe9-7dff-4d43-aa93-4c94deef063f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:41.352390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb66fe9-7dff-4d43-aa93-4c94deef063f None admin] PATH_INFO: `/v3/users/df2df47eaee249f190f66244e4a6c844` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:41.353665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb66fe9-7dff-4d43-aa93-4c94deef063f None admin] RBAC: Authorizing `identity:get_user(user_id=df2df47eaee249f190f66244e4a6c844)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:41.356037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb66fe9-7dff-4d43-aa93-4c94deef063f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:41.358057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 145/291] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:41 2022] GET /identity/v3/users/df2df47eaee249f190f66244e4a6c844 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:41.366240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:41.482765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:41.520558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc736aaa-1cfe-4e15-a50c-6321cded1ded None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:41.571110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797fd3c-a46f-4bcf-a66f-7c47727cf3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:41.571847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797fd3c-a46f-4bcf-a66f-7c47727cf3bd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:41.572131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797fd3c-a46f-4bcf-a66f-7c47727cf3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:41.572408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797fd3c-a46f-4bcf-a66f-7c47727cf3bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:41.654096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797fd3c-a46f-4bcf-a66f-7c47727cf3bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:41.655608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797fd3c-a46f-4bcf-a66f-7c47727cf3bd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:41.763716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:41.814879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 147/292] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:41.820488 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb66fe9-7dff-4d43-aa93-4c94deef063f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:41.965042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb66fe9-7dff-4d43-aa93-4c94deef063f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:42.006745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5a224-570e-4057-b358-b7131f940807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:42.007641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5a224-570e-4057-b358-b7131f940807 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.007922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5a224-570e-4057-b358-b7131f940807 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.008196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5a224-570e-4057-b358-b7131f940807 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.009404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5a224-570e-4057-b358-b7131f940807 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:42.010910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5a224-570e-4057-b358-b7131f940807 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:42.012800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 146/293] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:41 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:42.018863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797fd3c-a46f-4bcf-a66f-7c47727cf3bd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:42.121653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06045c-ef55-47f1-94a1-75d1f4b289d6 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.121971 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06045c-ef55-47f1-94a1-75d1f4b289d6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.122218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06045c-ef55-47f1-94a1-75d1f4b289d6 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.123652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 147/294] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:11:42.135259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea907a-34d3-494c-8d03-c0b219dfb433 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.135888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea907a-34d3-494c-8d03-c0b219dfb433 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.136354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea907a-34d3-494c-8d03-c0b219dfb433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.163640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797fd3c-a46f-4bcf-a66f-7c47727cf3bd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:42.172517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b8ea907a-34d3-494c-8d03-c0b219dfb433 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:42.173736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b8ea907a-34d3-494c-8d03-c0b219dfb433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:42.206903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a56519-3acd-41a0-9482-1047b2b760f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:42.207952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56519-3acd-41a0-9482-1047b2b760f5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.208461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56519-3acd-41a0-9482-1047b2b760f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.208936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56519-3acd-41a0-9482-1047b2b760f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.209981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a56519-3acd-41a0-9482-1047b2b760f5 None admin] RBAC: Adding query filter params (role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=044d245aa6804c66a019fc432a744109, user.id=df2df47eaee249f190f66244e4a6c844) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:42.210546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a56519-3acd-41a0-9482-1047b2b760f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:42.212249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a56519-3acd-41a0-9482-1047b2b760f5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:42.221135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 148/295] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:42 2022] GET /identity/v3/role_assignments?user.id=df2df47eaee249f190f66244e4a6c844&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:42.297860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:42.329232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 148/296] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:42.355588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1495e-94af-4cc6-b254-e982a6baa4fd None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.355898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1495e-94af-4cc6-b254-e982a6baa4fd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.356142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1495e-94af-4cc6-b254-e982a6baa4fd None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.357789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 149/297] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:42.363725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b2d47-29ba-421c-b25b-279bc06f8796 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.363998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b2d47-29ba-421c-b25b-279bc06f8796 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.364275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b2d47-29ba-421c-b25b-279bc06f8796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.369042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1495e-94af-4cc6-b254-e982a6baa4fd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:42.385989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-150b2d47-29ba-421c-b25b-279bc06f8796 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:42.387203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-150b2d47-29ba-421c-b25b-279bc06f8796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:42.462549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:42.525002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 149/298] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:42.548290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b2d47-29ba-421c-b25b-279bc06f8796 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:42.554789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1495e-94af-4cc6-b254-e982a6baa4fd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:42.608975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f88e7-57c1-4146-a314-283fc57e7513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:42.610435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f88e7-57c1-4146-a314-283fc57e7513 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.610893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f88e7-57c1-4146-a314-283fc57e7513 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.611589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f88e7-57c1-4146-a314-283fc57e7513 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.613175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f88e7-57c1-4146-a314-283fc57e7513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:42.615203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f88e7-57c1-4146-a314-283fc57e7513 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:42.617828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 150/299] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:42.624161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f88e7-57c1-4146-a314-283fc57e7513 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:42.676184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b2d47-29ba-421c-b25b-279bc06f8796 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:42.693031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:42.733897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b35dd-ead1-4655-a6c1-b6f8a5509402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:42.735014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b35dd-ead1-4655-a6c1-b6f8a5509402 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.735510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b35dd-ead1-4655-a6c1-b6f8a5509402 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.735960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b35dd-ead1-4655-a6c1-b6f8a5509402 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.752796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b35dd-ead1-4655-a6c1-b6f8a5509402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:42.762274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f88e7-57c1-4146-a314-283fc57e7513 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-925b35dd-ead1-4655-a6c1-b6f8a5509402 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:42.769854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:42.770822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:42.771636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:42.773270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 150/300] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:42.773768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:42.773937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a48d-7ec9-4e95-87d9-5c136a347be5 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.774220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a48d-7ec9-4e95-87d9-5c136a347be5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.774525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a48d-7ec9-4e95-87d9-5c136a347be5 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.776706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 151/301] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:42.778953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071a48d-7ec9-4e95-87d9-5c136a347be5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:42.812614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0f4f-27fb-4349-a71c-7ffe31469665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:42.813330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0f4f-27fb-4349-a71c-7ffe31469665 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.813664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0f4f-27fb-4349-a71c-7ffe31469665 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.813965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0f4f-27fb-4349-a71c-7ffe31469665 None admin] PATH_INFO: `/v3/groups` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.814734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0f4f-27fb-4349-a71c-7ffe31469665 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:42.816427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0f4f-27fb-4349-a71c-7ffe31469665 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:42.833920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 151/302] 10.222.0.33 () {60 vars in 1134 bytes} [Sat May 28 17:11:42 2022] POST /identity/v3/groups => generated 229 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:11:42.839491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f3fd1-74ad-464b-ac99-4a94fb1a0ccd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.839491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f3fd1-74ad-464b-ac99-4a94fb1a0ccd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.839491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f3fd1-74ad-464b-ac99-4a94fb1a0ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.870398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c29f3fd1-74ad-464b-ac99-4a94fb1a0ccd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:42.872066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c29f3fd1-74ad-464b-ac99-4a94fb1a0ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:42.912135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071a48d-7ec9-4e95-87d9-5c136a347be5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:42.957108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:42.958000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:42.958386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:42.958926 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:42.959842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:42.960256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:42.962126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:42.975558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 152/303] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:42.975959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 152/304] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:42.979465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f3fd1-74ad-464b-ac99-4a94fb1a0ccd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:43.001209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:43.047719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:43.103979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:43.110372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f3fd1-74ad-464b-ac99-4a94fb1a0ccd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:43.147388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14ad03-2ca0-44bc-a410-fbf76eddf150 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:43.151755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4cdc-c14e-40ad-844b-ab2f87a97ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:43.152704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4cdc-c14e-40ad-844b-ab2f87a97ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.153053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4cdc-c14e-40ad-844b-ab2f87a97ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.153393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4cdc-c14e-40ad-844b-ab2f87a97ff4 None admin] PATH_INFO: `/v3/users/d91324beaf684f7dbb17a9f4803f8e78` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.168833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc4cdc-c14e-40ad-844b-ab2f87a97ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=d91324beaf684f7dbb17a9f4803f8e78)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:43.170492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc4cdc-c14e-40ad-844b-ab2f87a97ff4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:43.172615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 153/305] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:42 2022] GET /identity/v3/users/d91324beaf684f7dbb17a9f4803f8e78 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:43.178467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4cdc-c14e-40ad-844b-ab2f87a97ff4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:43.187387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ee317-d5a6-4192-9db9-0cf2c1118e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:43.188300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ee317-d5a6-4192-9db9-0cf2c1118e77 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.189062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ee317-d5a6-4192-9db9-0cf2c1118e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.189337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ee317-d5a6-4192-9db9-0cf2c1118e77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.202908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ee317-d5a6-4192-9db9-0cf2c1118e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3e9ee317-d5a6-4192-9db9-0cf2c1118e77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:43.211425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:43.212373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:43.214364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 153/306] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:43.219311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ee317-d5a6-4192-9db9-0cf2c1118e77 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:43.294925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4cdc-c14e-40ad-844b-ab2f87a97ff4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:43.336706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:43.337298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.337579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.337960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.339408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] RBAC: Adding query filter params (role.id=7b21f490e8634fb09bf8b449b44a937a, user.id=d91324beaf684f7dbb17a9f4803f8e78, scope.system=all) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:43.339728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:43.344665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:43.346608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:43.348414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 154/307] 10.222.0.33 () {56 vars in 1308 bytes} [Sat May 28 17:11:43 2022] GET /identity/v3/role_assignments?user.id=d91324beaf684f7dbb17a9f4803f8e78&scope.system=all&role.id=7b21f490e8634fb09bf8b449b44a937a => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:43.355050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ee317-d5a6-4192-9db9-0cf2c1118e77 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:43.395111 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bf4df-b8d3-42e7-8d53-33b4970519b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:43.396253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf4df-b8d3-42e7-8d53-33b4970519b5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.396740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf4df-b8d3-42e7-8d53-33b4970519b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.397214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf4df-b8d3-42e7-8d53-33b4970519b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.398203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bf4df-b8d3-42e7-8d53-33b4970519b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:43.399108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bf4df-b8d3-42e7-8d53-33b4970519b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:43.400741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bf4df-b8d3-42e7-8d53-33b4970519b5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:43.411658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 154/308] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:43.417259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:43.440939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:43.518502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973b8b0-8217-4ee6-a60c-b598a93215bb None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.518829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973b8b0-8217-4ee6-a60c-b598a93215bb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.519375 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973b8b0-8217-4ee6-a60c-b598a93215bb None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.522567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 155/309] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:11:43.529022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93732-5579-413f-9f22-56c8b7195d3c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.529645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93732-5579-413f-9f22-56c8b7195d3c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.530138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93732-5579-413f-9f22-56c8b7195d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.538390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa12849-b729-4bc5-9ff1-f072000ddabb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:43.554079 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-81f93732-5579-413f-9f22-56c8b7195d3c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:43.556212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-81f93732-5579-413f-9f22-56c8b7195d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:43.577624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7888c3-c998-438e-bb7a-b76b59c7e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:43.578610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7888c3-c998-438e-bb7a-b76b59c7e0cd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.579010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7888c3-c998-438e-bb7a-b76b59c7e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.579282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7888c3-c998-438e-bb7a-b76b59c7e0cd None admin] PATH_INFO: `/v3/users/0807822c7b3d4c92b81c4fa42f387832` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.580513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7888c3-c998-438e-bb7a-b76b59c7e0cd None admin] RBAC: Authorizing `identity:get_user(user_id=0807822c7b3d4c92b81c4fa42f387832)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:43.581955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7888c3-c998-438e-bb7a-b76b59c7e0cd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:43.587443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 155/310] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:43 2022] GET /identity/v3/users/0807822c7b3d4c92b81c4fa42f387832 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:43.590138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7888c3-c998-438e-bb7a-b76b59c7e0cd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:43.656034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 156/311] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:43.680494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f93732-5579-413f-9f22-56c8b7195d3c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:43.686506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:43.702393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7888c3-c998-438e-bb7a-b76b59c7e0cd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:43.748840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eb821-e523-4f4a-9363-2077615e0d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:43.749711 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eb821-e523-4f4a-9363-2077615e0d3a None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.749983 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eb821-e523-4f4a-9363-2077615e0d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.750257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eb821-e523-4f4a-9363-2077615e0d3a None admin] PATH_INFO: `/v3/system/users/0807822c7b3d4c92b81c4fa42f387832/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.751815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40eb821-e523-4f4a-9363-2077615e0d3a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0807822c7b3d4c92b81c4fa42f387832, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:43.753192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40eb821-e523-4f4a-9363-2077615e0d3a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:43.767435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 156/312] 10.222.0.33 () {58 vars in 1334 bytes} [Sat May 28 17:11:43 2022] PUT /identity/v3/system/users/0807822c7b3d4c92b81c4fa42f387832/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:43.807948 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:43.825798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f93732-5579-413f-9f22-56c8b7195d3c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:43.866351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267796f-32ed-4ec9-979e-750973a92276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:43.867134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267796f-32ed-4ec9-979e-750973a92276 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:43.867447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267796f-32ed-4ec9-979e-750973a92276 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:43.867725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267796f-32ed-4ec9-979e-750973a92276 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:43.880315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267796f-32ed-4ec9-979e-750973a92276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-1267796f-32ed-4ec9-979e-750973a92276 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:43.888524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:43.889483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:43.890282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 157/313] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:11:43.900863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eb821-e523-4f4a-9363-2077615e0d3a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:44.013545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eb821-e523-4f4a-9363-2077615e0d3a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:44.051989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:44.053007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:44.053674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:44.054166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:44.055122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:44.055654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:44.057317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:44.066922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 157/314] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:44.073355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267796f-32ed-4ec9-979e-750973a92276 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:44.187097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267796f-32ed-4ec9-979e-750973a92276 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:44.222944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29451d9-0830-425c-a5ba-cca0b825f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:44.223850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29451d9-0830-425c-a5ba-cca0b825f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:44.224129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29451d9-0830-425c-a5ba-cca0b825f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:44.224397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29451d9-0830-425c-a5ba-cca0b825f0f9 None admin] PATH_INFO: `/v3/users/72e821ca8139466cbc5fcd065effa45c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:44.225665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29451d9-0830-425c-a5ba-cca0b825f0f9 None admin] RBAC: Authorizing `identity:get_user(user_id=72e821ca8139466cbc5fcd065effa45c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:44.227202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29451d9-0830-425c-a5ba-cca0b825f0f9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:44.229203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 158/315] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:44 2022] GET /identity/v3/users/72e821ca8139466cbc5fcd065effa45c => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:44.235007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:44.250932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:44.342849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d74dd-584a-4823-8d21-5707e7ed6102 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:44.378118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630e6d3-48fd-4c04-a466-f069ff7aae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:44.379079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630e6d3-48fd-4c04-a466-f069ff7aae0a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:44.379615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630e6d3-48fd-4c04-a466-f069ff7aae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:44.380095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630e6d3-48fd-4c04-a466-f069ff7aae0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:44.455185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630e6d3-48fd-4c04-a466-f069ff7aae0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:44.456644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630e6d3-48fd-4c04-a466-f069ff7aae0a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:44.471664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:44.563103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 158/316] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:44.568446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29451d9-0830-425c-a5ba-cca0b825f0f9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:44.589053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:44.681081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29451d9-0830-425c-a5ba-cca0b825f0f9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:44.723406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad199-5cf2-4865-b673-ca86cdc93cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:44.724536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad199-5cf2-4865-b673-ca86cdc93cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:44.725011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad199-5cf2-4865-b673-ca86cdc93cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:44.725495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad199-5cf2-4865-b673-ca86cdc93cc1 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:44.726913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ad199-5cf2-4865-b673-ca86cdc93cc1 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:44.728586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ad199-5cf2-4865-b673-ca86cdc93cc1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:44.731031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 159/317] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:44 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:44.736951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630e6d3-48fd-4c04-a466-f069ff7aae0a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:44.837377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630e6d3-48fd-4c04-a466-f069ff7aae0a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:44.851391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:44.873783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29015ac4-92b4-4f06-8f1d-495f812aa009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:44.874439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29015ac4-92b4-4f06-8f1d-495f812aa009 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:44.874719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29015ac4-92b4-4f06-8f1d-495f812aa009 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:44.874993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29015ac4-92b4-4f06-8f1d-495f812aa009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:44.875870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29015ac4-92b4-4f06-8f1d-495f812aa009 None admin] RBAC: Adding query filter params (role.id=7b21f490e8634fb09bf8b449b44a937a, scope.project.id=044d245aa6804c66a019fc432a744109, user.id=72e821ca8139466cbc5fcd065effa45c) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:44.876181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29015ac4-92b4-4f06-8f1d-495f812aa009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:44.877617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29015ac4-92b4-4f06-8f1d-495f812aa009 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:44.886117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 159/318] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:11:44 2022] GET /identity/v3/role_assignments?user.id=72e821ca8139466cbc5fcd065effa45c&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=7b21f490e8634fb09bf8b449b44a937a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:45.146338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f95f0-9c1d-4ccf-ba54-589a69c825f8 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:45.146824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f95f0-9c1d-4ccf-ba54-589a69c825f8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:45.146824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f95f0-9c1d-4ccf-ba54-589a69c825f8 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:45.148474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 160/319] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:45.154483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9690-632e-489d-82ff-5834fe483dfb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:45.154796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9690-632e-489d-82ff-5834fe483dfb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:45.155052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9690-632e-489d-82ff-5834fe483dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:45.174383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fcda9690-632e-489d-82ff-5834fe483dfb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:45.175908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fcda9690-632e-489d-82ff-5834fe483dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:45.245426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 160/320] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:45.268500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f95f0-9c1d-4ccf-ba54-589a69c825f8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:45.355967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:45.374937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f95f0-9c1d-4ccf-ba54-589a69c825f8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:45.408676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2a88d-beca-40b2-8eb5-27cca116752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:45.409381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a88d-beca-40b2-8eb5-27cca116752e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:45.409661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a88d-beca-40b2-8eb5-27cca116752e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:45.409935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a88d-beca-40b2-8eb5-27cca116752e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:45.411083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2a88d-beca-40b2-8eb5-27cca116752e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:45.412547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2a88d-beca-40b2-8eb5-27cca116752e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:45.414397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 161/321] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:45.419960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9690-632e-489d-82ff-5834fe483dfb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:45.526787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9690-632e-489d-82ff-5834fe483dfb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:45.534465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:45.563257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c589bf7-d66f-4c3d-a530-b6c0e645edf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:45.563967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c589bf7-d66f-4c3d-a530-b6c0e645edf6 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:45.564253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c589bf7-d66f-4c3d-a530-b6c0e645edf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:45.564532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c589bf7-d66f-4c3d-a530-b6c0e645edf6 None admin] PATH_INFO: `/v3/groups` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:45.565236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c589bf7-d66f-4c3d-a530-b6c0e645edf6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:45.566681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c589bf7-d66f-4c3d-a530-b6c0e645edf6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:45.587373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 161/322] 10.222.0.33 () {60 vars in 1134 bytes} [Sat May 28 17:11:45 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:11:45.672524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd230f-7041-41b2-a7a8-ebc0c6709c72 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:45.672714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd230f-7041-41b2-a7a8-ebc0c6709c72 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:45.672997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd230f-7041-41b2-a7a8-ebc0c6709c72 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:45.674631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 162/323] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:45.680194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cca8c-97aa-428f-a97e-38ec3367ec72 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:45.680511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cca8c-97aa-428f-a97e-38ec3367ec72 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:45.680763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cca8c-97aa-428f-a97e-38ec3367ec72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:45.699107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b68cca8c-97aa-428f-a97e-38ec3367ec72 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:45.700356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b68cca8c-97aa-428f-a97e-38ec3367ec72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:45.782060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 162/324] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:45.805343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd230f-7041-41b2-a7a8-ebc0c6709c72 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:45.924393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd230f-7041-41b2-a7a8-ebc0c6709c72 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:45.965068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c75b60-3ae3-4e1d-8e1b-4d1b0ccd75bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:45.965858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c75b60-3ae3-4e1d-8e1b-4d1b0ccd75bf None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:45.966138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c75b60-3ae3-4e1d-8e1b-4d1b0ccd75bf None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:45.966414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c75b60-3ae3-4e1d-8e1b-4d1b0ccd75bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:45.977489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c75b60-3ae3-4e1d-8e1b-4d1b0ccd75bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e7c75b60-3ae3-4e1d-8e1b-4d1b0ccd75bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:45.986421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:45.987385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:45.988233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 163/325] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:45.993473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cca8c-97aa-428f-a97e-38ec3367ec72 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:46.103989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:46.112297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cca8c-97aa-428f-a97e-38ec3367ec72 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:46.116081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2170a8-2845-45a8-a0b8-0c3b24d64ea5 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:46.117225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2170a8-2845-45a8-a0b8-0c3b24d64ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:46.117225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2170a8-2845-45a8-a0b8-0c3b24d64ea5 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:46.118257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 164/326] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:46.124392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d361-cca2-4b1e-b31b-f923a563b9a6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:46.124678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d361-cca2-4b1e-b31b-f923a563b9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:46.124964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d361-cca2-4b1e-b31b-f923a563b9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:46.147741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f4f3d361-cca2-4b1e-b31b-f923a563b9a6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:46.148917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f4f3d361-cca2-4b1e-b31b-f923a563b9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:46.158581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:46.162741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:46.164381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:46.164678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:46.164958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:46.165722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:46.166031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:46.168294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:46.180799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 163/327] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:46.186474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:46.240399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 165/328] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:46.266468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3d361-cca2-4b1e-b31b-f923a563b9a6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:46.317094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde88d4-1f7d-411f-8b37-509d6af3bf9a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:46.356541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3be4e-5253-4f45-8523-05163c3438e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:46.357781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3be4e-5253-4f45-8523-05163c3438e1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:46.358531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3be4e-5253-4f45-8523-05163c3438e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:46.359008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3be4e-5253-4f45-8523-05163c3438e1 None admin] PATH_INFO: `/v3/users/d91324beaf684f7dbb17a9f4803f8e78` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:46.360911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3be4e-5253-4f45-8523-05163c3438e1 None admin] RBAC: Authorizing `identity:get_user(user_id=d91324beaf684f7dbb17a9f4803f8e78)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:46.362567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3be4e-5253-4f45-8523-05163c3438e1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:46.364930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 164/329] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:46 2022] GET /identity/v3/users/d91324beaf684f7dbb17a9f4803f8e78 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:46.370080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3be4e-5253-4f45-8523-05163c3438e1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:46.396908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3d361-cca2-4b1e-b31b-f923a563b9a6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:46.410145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:46.443713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b163d68-066e-403c-aa6e-24277ff5d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:46.444849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b163d68-066e-403c-aa6e-24277ff5d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:46.445330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b163d68-066e-403c-aa6e-24277ff5d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:46.445765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b163d68-066e-403c-aa6e-24277ff5d2f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:46.463495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b163d68-066e-403c-aa6e-24277ff5d2f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-7b163d68-066e-403c-aa6e-24277ff5d2f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:46.472516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:46.473441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:46.474224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:46.476886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 166/330] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:11:46.483582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b163d68-066e-403c-aa6e-24277ff5d2f0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:46.487136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:46.506159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3be4e-5253-4f45-8523-05163c3438e1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:46.549018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4060ce-a64a-4ca4-b169-6abfbb7d32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:46.550340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4060ce-a64a-4ca4-b169-6abfbb7d32de None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:46.550945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4060ce-a64a-4ca4-b169-6abfbb7d32de None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:46.551451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4060ce-a64a-4ca4-b169-6abfbb7d32de None admin] PATH_INFO: `/v3/system/users/d91324beaf684f7dbb17a9f4803f8e78/roles/7b21f490e8634fb09bf8b449b44a937a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:46.553202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4060ce-a64a-4ca4-b169-6abfbb7d32de None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d91324beaf684f7dbb17a9f4803f8e78, role_id=7b21f490e8634fb09bf8b449b44a937a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:46.554964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4060ce-a64a-4ca4-b169-6abfbb7d32de None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:46.565624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 165/331] 10.222.0.33 () {58 vars in 1334 bytes} [Sat May 28 17:11:46 2022] PUT /identity/v3/system/users/d91324beaf684f7dbb17a9f4803f8e78/roles/7b21f490e8634fb09bf8b449b44a937a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:46.622151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b163d68-066e-403c-aa6e-24277ff5d2f0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:46.670327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:46.671434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:46.671929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:46.672418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:46.673457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:46.673981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:46.675737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:46.692677 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4060ce-a64a-4ca4-b169-6abfbb7d32de None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:46.693458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 167/332] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:46.804102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:46.822033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4060ce-a64a-4ca4-b169-6abfbb7d32de None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:46.870841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cebbb7-65e9-43dd-874c-c21e46743075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:46.871766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cebbb7-65e9-43dd-874c-c21e46743075 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:46.872047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cebbb7-65e9-43dd-874c-c21e46743075 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:46.872319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cebbb7-65e9-43dd-874c-c21e46743075 None admin] PATH_INFO: `/v3/users/0807822c7b3d4c92b81c4fa42f387832` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:46.873529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cebbb7-65e9-43dd-874c-c21e46743075 None admin] RBAC: Authorizing `identity:get_user(user_id=0807822c7b3d4c92b81c4fa42f387832)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:46.874961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cebbb7-65e9-43dd-874c-c21e46743075 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:46.878687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 166/333] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:46 2022] GET /identity/v3/users/0807822c7b3d4c92b81c4fa42f387832 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:46.884190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:46.966860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:47.012977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f564d0e-a05b-4efb-8788-55dca4a4d975 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:47.055408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dde5d-076d-4f00-beb7-6af17bb9c117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:47.056077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dde5d-076d-4f00-beb7-6af17bb9c117 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:47.056358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dde5d-076d-4f00-beb7-6af17bb9c117 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:47.056626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dde5d-076d-4f00-beb7-6af17bb9c117 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:47.057535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dde5d-076d-4f00-beb7-6af17bb9c117 None admin] RBAC: Adding query filter params (role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, user.id=0807822c7b3d4c92b81c4fa42f387832, scope.system=all) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:47.057933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dde5d-076d-4f00-beb7-6af17bb9c117 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:47.059391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dde5d-076d-4f00-beb7-6af17bb9c117 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:47.076303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 168/334] 10.222.0.33 () {56 vars in 1308 bytes} [Sat May 28 17:11:46 2022] GET /identity/v3/role_assignments?user.id=0807822c7b3d4c92b81c4fa42f387832&scope.system=all&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:48.695771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd38ac-8ff9-4ff6-8a67-f3e62d16f990 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:48.696286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd38ac-8ff9-4ff6-8a67-f3e62d16f990 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:48.696391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd38ac-8ff9-4ff6-8a67-f3e62d16f990 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:48.698480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 167/335] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:48.704280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d829a-d858-4706-8e61-dee3b192be84 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:48.704812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d829a-d858-4706-8e61-dee3b192be84 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:48.704812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d829a-d858-4706-8e61-dee3b192be84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:48.731063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-254d829a-d858-4706-8e61-dee3b192be84 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:48.732421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-254d829a-d858-4706-8e61-dee3b192be84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:48.828165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 169/336] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:48.876891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd38ac-8ff9-4ff6-8a67-f3e62d16f990 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:49.009056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd38ac-8ff9-4ff6-8a67-f3e62d16f990 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:49.050779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abad4d-887e-4496-8b69-9769a5167765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:49.051615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abad4d-887e-4496-8b69-9769a5167765 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:49.051891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abad4d-887e-4496-8b69-9769a5167765 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:49.052160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abad4d-887e-4496-8b69-9769a5167765 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:49.059377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5162c-d288-4fe8-bbdd-28a01b7be9ea None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:49.059879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5162c-d288-4fe8-bbdd-28a01b7be9ea None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:49.060319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5162c-d288-4fe8-bbdd-28a01b7be9ea None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:49.062077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 170/337] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:49.069041 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abad4d-887e-4496-8b69-9769a5167765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:49.069336 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd994e6-19be-48a1-91e4-70f51b25cc4b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:49.069575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd994e6-19be-48a1-91e4-70f51b25cc4b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:49.069857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd994e6-19be-48a1-91e4-70f51b25cc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c4abad4d-887e-4496-8b69-9769a5167765 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:49.077851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:49.078750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:49.079584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:49.080198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 168/338] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:49.086705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abad4d-887e-4496-8b69-9769a5167765 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:49.100191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dfd994e6-19be-48a1-91e4-70f51b25cc4b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:49.101401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dfd994e6-19be-48a1-91e4-70f51b25cc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:49.120018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:49.234420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abad4d-887e-4496-8b69-9769a5167765 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:49.239680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:49.284755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:49.285375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:49.285671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:49.285940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:49.286706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:49.287098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:49.288543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:49.294530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 171/339] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:49.305508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 169/340] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:49.311316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd994e6-19be-48a1-91e4-70f51b25cc4b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:49.330464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:49.475185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd994e6-19be-48a1-91e4-70f51b25cc4b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:49.530773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a54924-c150-491b-a1c1-845f108ba94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:49.531466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a54924-c150-491b-a1c1-845f108ba94f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:49.531745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a54924-c150-491b-a1c1-845f108ba94f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:49.532013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a54924-c150-491b-a1c1-845f108ba94f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:49.545693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:49.610373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:49.613537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af575ea-399b-45c5-8c8b-2178d5b575df None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:49.632873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a54924-c150-491b-a1c1-845f108ba94f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:49.634407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a54924-c150-491b-a1c1-845f108ba94f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:49.704633 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73486b-f935-4e13-aad4-717948fef0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:49.705470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73486b-f935-4e13-aad4-717948fef0bf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:49.705752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73486b-f935-4e13-aad4-717948fef0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:49.706030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73486b-f935-4e13-aad4-717948fef0bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:49.740816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73486b-f935-4e13-aad4-717948fef0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:49.743719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 172/341] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:49.753359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a54924-c150-491b-a1c1-845f108ba94f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-af73486b-f935-4e13-aad4-717948fef0bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:49.763422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:49.764546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:49.765678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:49.773317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 170/342] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:49.781798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73486b-f935-4e13-aad4-717948fef0bf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:49.875595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:49.898937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a54924-c150-491b-a1c1-845f108ba94f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:49.943601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb05dd9-8dd5-4a7f-91f9-7a153925e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:49.944872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb05dd9-8dd5-4a7f-91f9-7a153925e02c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:49.945314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb05dd9-8dd5-4a7f-91f9-7a153925e02c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:49.945723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb05dd9-8dd5-4a7f-91f9-7a153925e02c None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:49.948693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb05dd9-8dd5-4a7f-91f9-7a153925e02c None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:49.950599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb05dd9-8dd5-4a7f-91f9-7a153925e02c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:49.953322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 173/343] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:49 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:49.963162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb05dd9-8dd5-4a7f-91f9-7a153925e02c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:50.042218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73486b-f935-4e13-aad4-717948fef0bf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:50.091143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb05dd9-8dd5-4a7f-91f9-7a153925e02c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:50.134454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa2dd1-d328-4a85-b994-ebeac5df2e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:50.137004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2dd1-d328-4a85-b994-ebeac5df2e10 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.137777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07bcf9-0a7c-46ce-a363-9aa424c2c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:50.138801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07bcf9-0a7c-46ce-a363-9aa424c2c511 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.139229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2dd1-d328-4a85-b994-ebeac5df2e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.139658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2dd1-d328-4a85-b994-ebeac5df2e10 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.140084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07bcf9-0a7c-46ce-a363-9aa424c2c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.140994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa2dd1-d328-4a85-b994-ebeac5df2e10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:50.141416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07bcf9-0a7c-46ce-a363-9aa424c2c511 None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.142422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa2dd1-d328-4a85-b994-ebeac5df2e10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:50.145373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa2dd1-d328-4a85-b994-ebeac5df2e10 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:50.156405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07bcf9-0a7c-46ce-a363-9aa424c2c511 None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:50.158106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07bcf9-0a7c-46ce-a363-9aa424c2c511 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:50.162058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 174/344] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:49 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:50.163561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e2ef-d5a1-42ef-8017-e2ed7ac6c33a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.164008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e2ef-d5a1-42ef-8017-e2ed7ac6c33a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.164287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 171/345] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:50.164594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e2ef-d5a1-42ef-8017-e2ed7ac6c33a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.165709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104127f-abc5-4a2c-844b-7cf56313a65b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.166003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104127f-abc5-4a2c-844b-7cf56313a65b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.166280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104127f-abc5-4a2c-844b-7cf56313a65b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.167035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 175/346] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:50.171498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 172/347] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:50.172525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136e2ef-d5a1-42ef-8017-e2ed7ac6c33a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:50.173714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b634eea-fbf3-46a7-b44e-60fb4485ffae None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.174014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b634eea-fbf3-46a7-b44e-60fb4485ffae None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.174287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b634eea-fbf3-46a7-b44e-60fb4485ffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.180703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:50.198872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3b634eea-fbf3-46a7-b44e-60fb4485ffae None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:50.199294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:50.200171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3b634eea-fbf3-46a7-b44e-60fb4485ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:50.292834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 173/348] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:50.294291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b634eea-fbf3-46a7-b44e-60fb4485ffae None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:50.327375 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136e2ef-d5a1-42ef-8017-e2ed7ac6c33a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:50.364868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d58e3-ad4a-483e-acc4-b1c13c9e45b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:50.365461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d58e3-ad4a-483e-acc4-b1c13c9e45b0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.365747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d58e3-ad4a-483e-acc4-b1c13c9e45b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.366026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d58e3-ad4a-483e-acc4-b1c13c9e45b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.425674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b634eea-fbf3-46a7-b44e-60fb4485ffae None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:50.448553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d58e3-ad4a-483e-acc4-b1c13c9e45b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:50.450165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d58e3-ad4a-483e-acc4-b1c13c9e45b0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:50.459759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:50.468898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:50.469073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ba54a-ee7d-4000-9136-0e296e60588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:50.469894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ba54a-ee7d-4000-9136-0e296e60588d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.470415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ba54a-ee7d-4000-9136-0e296e60588d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.470614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ba54a-ee7d-4000-9136-0e296e60588d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.474998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ba54a-ee7d-4000-9136-0e296e60588d None admin] RBAC: Adding query filter params (group.id=95d6275f34fb4208a23e42ed2528cd3e, role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=7b05c4db805c48ef88067735bdedc47e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:50.474998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ba54a-ee7d-4000-9136-0e296e60588d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:50.474998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ba54a-ee7d-4000-9136-0e296e60588d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:50.482032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 174/349] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:11:50 2022] GET /identity/v3/role_assignments?group.id=95d6275f34fb4208a23e42ed2528cd3e&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:50.494059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aca340-0581-48ec-93a7-65775693ff0d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.494421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aca340-0581-48ec-93a7-65775693ff0d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.494754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aca340-0581-48ec-93a7-65775693ff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.512691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f4aca340-0581-48ec-93a7-65775693ff0d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:50.513914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f4aca340-0581-48ec-93a7-65775693ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:50.581857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 176/350] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:50.583108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d58e3-ad4a-483e-acc4-b1c13c9e45b0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:50.603697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 175/351] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:50.603697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2efab4-9c4e-4d2f-a6a4-5455f313afc0 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.603697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2efab4-9c4e-4d2f-a6a4-5455f313afc0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.603697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2efab4-9c4e-4d2f-a6a4-5455f313afc0 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.603697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 176/352] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:50.605987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2efab4-9c4e-4d2f-a6a4-5455f313afc0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:50.725691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2efab4-9c4e-4d2f-a6a4-5455f313afc0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:50.746703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d58e3-ad4a-483e-acc4-b1c13c9e45b0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:50.764176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:50.764790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:50.777336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a86702-0373-4488-869c-671bca574765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:50.778857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a86702-0373-4488-869c-671bca574765 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.779592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a86702-0373-4488-869c-671bca574765 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.780011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a86702-0373-4488-869c-671bca574765 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.781594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a86702-0373-4488-869c-671bca574765 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:50.783806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a86702-0373-4488-869c-671bca574765 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:50.786828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 177/353] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:50 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:50.788246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be369b-1ec7-4a82-9780-44178f2ba0d9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.788678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be369b-1ec7-4a82-9780-44178f2ba0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.789095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be369b-1ec7-4a82-9780-44178f2ba0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.791855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6f378-5ba4-48f6-935c-cf49bd4c63d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:50.792847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f378-5ba4-48f6-935c-cf49bd4c63d7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.793244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f378-5ba4-48f6-935c-cf49bd4c63d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.793643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f378-5ba4-48f6-935c-cf49bd4c63d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:50.808343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6f378-5ba4-48f6-935c-cf49bd4c63d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:50.809284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-26be369b-1ec7-4a82-9780-44178f2ba0d9 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:50.810614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-26be369b-1ec7-4a82-9780-44178f2ba0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-91f6f378-5ba4-48f6-935c-cf49bd4c63d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:50.819237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:50.820365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:50.821389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 177/354] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:50.822099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6f378-5ba4-48f6-935c-cf49bd4c63d7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:50.906985 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 178/355] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:50.908128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be369b-1ec7-4a82-9780-44178f2ba0d9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:50.945136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6f378-5ba4-48f6-935c-cf49bd4c63d7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:50.987319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fea69-b4cc-4d36-867b-9c276b01d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:50.988641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fea69-b4cc-4d36-867b-9c276b01d398 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:50.989125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fea69-b4cc-4d36-867b-9c276b01d398 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:50.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fea69-b4cc-4d36-867b-9c276b01d398 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.004145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fea69-b4cc-4d36-867b-9c276b01d398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-153fea69-b4cc-4d36-867b-9c276b01d398 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:51.013156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:51.014092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:51.014898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 178/356] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:51.017134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfade3-a5d8-4169-9e33-4ad4d18da16a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.017134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfade3-a5d8-4169-9e33-4ad4d18da16a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.017134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfade3-a5d8-4169-9e33-4ad4d18da16a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.019173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 179/357] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:51.019935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfade3-a5d8-4169-9e33-4ad4d18da16a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.029707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be369b-1ec7-4a82-9780-44178f2ba0d9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.056447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:51.069723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:51.071136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7bf51-0383-4676-8d8b-4fa34b115730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.072053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7bf51-0383-4676-8d8b-4fa34b115730 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.072311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7bf51-0383-4676-8d8b-4fa34b115730 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.073499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7bf51-0383-4676-8d8b-4fa34b115730 None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.074637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7bf51-0383-4676-8d8b-4fa34b115730 None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:51.076141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7bf51-0383-4676-8d8b-4fa34b115730 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:51.077973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 179/358] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:50 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:51.078521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7bf51-0383-4676-8d8b-4fa34b115730 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.158086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfade3-a5d8-4169-9e33-4ad4d18da16a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.203922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.204935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.205419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.205913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.206936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:51.207574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:51.209232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:51.211145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7bf51-0383-4676-8d8b-4fa34b115730 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.221353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 180/359] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:51.222054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.250719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a76012-8a45-43c0-91f9-daa1126fdff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.251559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76012-8a45-43c0-91f9-daa1126fdff3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.251829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76012-8a45-43c0-91f9-daa1126fdff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.252157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76012-8a45-43c0-91f9-daa1126fdff3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.264368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a76012-8a45-43c0-91f9-daa1126fdff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-45a76012-8a45-43c0-91f9-daa1126fdff3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:51.272937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:51.273833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:51.274635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 180/360] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:51.275985 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dee22-6670-4d2a-86bb-c626bc9a83b6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.277758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dee22-6670-4d2a-86bb-c626bc9a83b6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.277758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dee22-6670-4d2a-86bb-c626bc9a83b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.297736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-755dee22-6670-4d2a-86bb-c626bc9a83b6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:51.298926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-755dee22-6670-4d2a-86bb-c626bc9a83b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:51.347550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:51.353862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c2245-6799-458f-9817-f19495c3c955 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.359918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:51.385366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 181/361] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:51.386221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755dee22-6670-4d2a-86bb-c626bc9a83b6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.398607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.399278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.400308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.400653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.402181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:51.402726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:51.403958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:51.416583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 181/362] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:51.417425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.522458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755dee22-6670-4d2a-86bb-c626bc9a83b6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.557784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f62d4-4825-47c1-943f-6c469883128f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.564096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.564815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.565101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.565423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.566684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] RBAC: Adding query filter params (group.id=95d6275f34fb4208a23e42ed2528cd3e, role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=7b05c4db805c48ef88067735bdedc47e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:51.567001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:51.569127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:51.577286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 182/363] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/role_assignments?group.id=95d6275f34fb4208a23e42ed2528cd3e&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:51.578409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.606529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430381c-93a4-4efd-a4e9-340d5b4c2f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.607614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430381c-93a4-4efd-a4e9-340d5b4c2f43 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.607899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430381c-93a4-4efd-a4e9-340d5b4c2f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.608175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430381c-93a4-4efd-a4e9-340d5b4c2f43 None admin] PATH_INFO: `/v3/users/d91324beaf684f7dbb17a9f4803f8e78` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.610118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430381c-93a4-4efd-a4e9-340d5b4c2f43 None admin] RBAC: Authorizing `identity:get_user(user_id=d91324beaf684f7dbb17a9f4803f8e78)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:51.612333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430381c-93a4-4efd-a4e9-340d5b4c2f43 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:51.614438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 182/364] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/users/d91324beaf684f7dbb17a9f4803f8e78 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:51.616685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430381c-93a4-4efd-a4e9-340d5b4c2f43 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.639452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:51.647982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:51.707802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21016155-e044-4e7a-9e72-7a9b3d2ead54 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.738567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430381c-93a4-4efd-a4e9-340d5b4c2f43 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.763013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.764526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.765104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.765583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.766617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:51.767220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:51.769164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:51.780033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8827fd-c7d9-4985-9632-6dbb7c7bdb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.780922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8827fd-c7d9-4985-9632-6dbb7c7bdb66 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.781303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8827fd-c7d9-4985-9632-6dbb7c7bdb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.781678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8827fd-c7d9-4985-9632-6dbb7c7bdb66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.788394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 183/365] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:51.788394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.805894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8827fd-c7d9-4985-9632-6dbb7c7bdb66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-aa8827fd-c7d9-4985-9632-6dbb7c7bdb66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:51.814746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:51.815770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:51.816568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 183/366] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:51.817231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8827fd-c7d9-4985-9632-6dbb7c7bdb66 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:51.930978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08f63c-0ad3-46e6-9386-ff0b3e726578 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.934350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:51.944516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:51.955472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8827fd-c7d9-4985-9632-6dbb7c7bdb66 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:51.981938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aa079-ca58-4d1d-a750-96533bce5476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.983184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aa079-ca58-4d1d-a750-96533bce5476 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.983838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aa079-ca58-4d1d-a750-96533bce5476 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.984225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aa079-ca58-4d1d-a750-96533bce5476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.995479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:51.996623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:51.997101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:51.997555 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:51.998707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] RBAC: Adding query filter params (role.id=7b21f490e8634fb09bf8b449b44a937a, user.id=d91324beaf684f7dbb17a9f4803f8e78, scope.system=all) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:51.999195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:52.000946 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:52.016945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 184/367] 10.222.0.33 () {56 vars in 1308 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/role_assignments?user.id=d91324beaf684f7dbb17a9f4803f8e78&scope.system=all&role.id=7b21f490e8634fb09bf8b449b44a937a => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:52.018126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:52.081186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8aa079-ca58-4d1d-a750-96533bce5476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:52.083222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8aa079-ca58-4d1d-a750-96533bce5476 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:52.234555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 184/368] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:52.235232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aa079-ca58-4d1d-a750-96533bce5476 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:52.254551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:52.262740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12b8bc-5ef2-439e-883a-7bf5f3ec3161 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:52.321609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0fb53-56b5-4aa7-ab5b-928638ce0943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:52.322281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0fb53-56b5-4aa7-ab5b-928638ce0943 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:52.322935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0fb53-56b5-4aa7-ab5b-928638ce0943 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:52.323518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0fb53-56b5-4aa7-ab5b-928638ce0943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:52.342745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:52.404537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aa079-ca58-4d1d-a750-96533bce5476 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:52.412448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0fb53-56b5-4aa7-ab5b-928638ce0943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:52.413935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0fb53-56b5-4aa7-ab5b-928638ce0943 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:52.453161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:52.454746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:52.455030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:52.455304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:52.456096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:52.456405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:52.457796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:52.471571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 185/369] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:52.472512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:52.536203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 185/370] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:52.537256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0fb53-56b5-4aa7-ab5b-928638ce0943 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:52.575627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:52.596550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:52.626060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588aa5b-c78f-45c5-b97d-2f38a0de29ae None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:52.665190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4879d7-b0b9-4c75-af46-1dc1b65804c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:52.666060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4879d7-b0b9-4c75-af46-1dc1b65804c2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:52.666392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4879d7-b0b9-4c75-af46-1dc1b65804c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:52.666732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4879d7-b0b9-4c75-af46-1dc1b65804c2 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:52.667897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4879d7-b0b9-4c75-af46-1dc1b65804c2 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:52.669644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4879d7-b0b9-4c75-af46-1dc1b65804c2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:52.671160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0fb53-56b5-4aa7-ab5b-928638ce0943 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:52.671387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 186/371] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:52 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:52.672150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4879d7-b0b9-4c75-af46-1dc1b65804c2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:52.723896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba391e0-92fc-4b51-8598-0f7edf5311b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:52.728248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba391e0-92fc-4b51-8598-0f7edf5311b7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:52.728248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba391e0-92fc-4b51-8598-0f7edf5311b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:52.728248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba391e0-92fc-4b51-8598-0f7edf5311b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:52.797279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4879d7-b0b9-4c75-af46-1dc1b65804c2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:52.819797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba391e0-92fc-4b51-8598-0f7edf5311b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:52.822031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba391e0-92fc-4b51-8598-0f7edf5311b7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:52.842373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5313e13-a6c5-4917-8589-ce289ce03dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:52.843282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5313e13-a6c5-4917-8589-ce289ce03dfe None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:52.844482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5313e13-a6c5-4917-8589-ce289ce03dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:52.845528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5313e13-a6c5-4917-8589-ce289ce03dfe None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:52.848218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5313e13-a6c5-4917-8589-ce289ce03dfe None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:52.849593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5313e13-a6c5-4917-8589-ce289ce03dfe None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:52.857255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 187/372] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:52 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:52.858121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5313e13-a6c5-4917-8589-ce289ce03dfe None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:52.867794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:52.887156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:52.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 186/373] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:52.972304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba391e0-92fc-4b51-8598-0f7edf5311b7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:52.987947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5313e13-a6c5-4917-8589-ce289ce03dfe None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:53.031300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518780fa-28f7-47a7-abd9-44ee47cd8648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:53.032199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518780fa-28f7-47a7-abd9-44ee47cd8648 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.032476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518780fa-28f7-47a7-abd9-44ee47cd8648 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.032749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518780fa-28f7-47a7-abd9-44ee47cd8648 None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.034409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518780fa-28f7-47a7-abd9-44ee47cd8648 None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:53.035688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518780fa-28f7-47a7-abd9-44ee47cd8648 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:53.043132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 188/374] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:52 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:53.044039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518780fa-28f7-47a7-abd9-44ee47cd8648 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:53.105720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba391e0-92fc-4b51-8598-0f7edf5311b7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:53.145408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:53.151448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d968be-2724-4ae7-b7c9-365396289257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:53.152546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d968be-2724-4ae7-b7c9-365396289257 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.152819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d968be-2724-4ae7-b7c9-365396289257 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.153087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d968be-2724-4ae7-b7c9-365396289257 None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.154283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d968be-2724-4ae7-b7c9-365396289257 None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:53.155835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d968be-2724-4ae7-b7c9-365396289257 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:53.157566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 187/375] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:52 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:53.158965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d968be-2724-4ae7-b7c9-365396289257 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:53.175442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:53.184980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518780fa-28f7-47a7-abd9-44ee47cd8648 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:53.223201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66408f08-749a-48ce-8c9d-eb7490bed868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:53.224092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66408f08-749a-48ce-8c9d-eb7490bed868 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.224369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66408f08-749a-48ce-8c9d-eb7490bed868 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.224649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66408f08-749a-48ce-8c9d-eb7490bed868 None admin] PATH_INFO: `/v3/projects/3b7d62044f5549a6b53772308a090b05` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.225823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66408f08-749a-48ce-8c9d-eb7490bed868 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7d62044f5549a6b53772308a090b05)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:53.227254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66408f08-749a-48ce-8c9d-eb7490bed868 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:53.231914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 189/376] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:53 2022] GET /identity/v3/projects/3b7d62044f5549a6b53772308a090b05 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:53.233173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32bc13-0718-477b-85ea-07533a127836 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.233709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32bc13-0718-477b-85ea-07533a127836 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.233986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32bc13-0718-477b-85ea-07533a127836 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.235323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 190/377] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:53.235899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32bc13-0718-477b-85ea-07533a127836 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:53.284205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d968be-2724-4ae7-b7c9-365396289257 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:53.327912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:53.328837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.329219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.329597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.330666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] RBAC: Adding query filter params (group.id=95d6275f34fb4208a23e42ed2528cd3e, role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=044d245aa6804c66a019fc432a744109) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:53.331108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:53.332687 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:53.339908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 188/378] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:11:53 2022] GET /identity/v3/role_assignments?group.id=95d6275f34fb4208a23e42ed2528cd3e&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:53.340910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:53.373731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32bc13-0718-477b-85ea-07533a127836 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:53.418309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f478fb-0873-490b-9edb-40ffbe819d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:53.419373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f478fb-0873-490b-9edb-40ffbe819d4b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.419801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f478fb-0873-490b-9edb-40ffbe819d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.420350 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f478fb-0873-490b-9edb-40ffbe819d4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.421202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f478fb-0873-490b-9edb-40ffbe819d4b None admin] RBAC: Adding query filter params (group.id=95d6275f34fb4208a23e42ed2528cd3e, role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=044d245aa6804c66a019fc432a744109) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:53.421519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f478fb-0873-490b-9edb-40ffbe819d4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:53.422971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f478fb-0873-490b-9edb-40ffbe819d4b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:53.435896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:53.438397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 191/379] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:11:53 2022] GET /identity/v3/role_assignments?group.id=95d6275f34fb4208a23e42ed2528cd3e&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:53.439985 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8447-a507-4de5-b6e3-a9efcc03b579 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.440276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8447-a507-4de5-b6e3-a9efcc03b579 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.440556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8447-a507-4de5-b6e3-a9efcc03b579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.465787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:53.474177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-36ce8447-a507-4de5-b6e3-a9efcc03b579 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:53.477579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-36ce8447-a507-4de5-b6e3-a9efcc03b579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:53.485588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72c116-469f-4b60-86a6-4a69686618f8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:53.572169 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc524ef-6862-464f-ad85-0ce46cc7b19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:53.575267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc524ef-6862-464f-ad85-0ce46cc7b19c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.575777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc524ef-6862-464f-ad85-0ce46cc7b19c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.576234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc524ef-6862-464f-ad85-0ce46cc7b19c None admin] PATH_INFO: `/v3/groups/32af467cae1d4edf9e86453f0a0cb312` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.589857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc524ef-6862-464f-ad85-0ce46cc7b19c None admin] RBAC: Authorizing `identity:get_group(group_id=32af467cae1d4edf9e86453f0a0cb312)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:53.591637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc524ef-6862-464f-ad85-0ce46cc7b19c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:53.597763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 189/380] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:53 2022] GET /identity/v3/groups/32af467cae1d4edf9e86453f0a0cb312 => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:53.604081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc524ef-6862-464f-ad85-0ce46cc7b19c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:53.610475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 192/381] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:53.637603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce8447-a507-4de5-b6e3-a9efcc03b579 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:53.740517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc524ef-6862-464f-ad85-0ce46cc7b19c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:53.781376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce8447-a507-4de5-b6e3-a9efcc03b579 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:53.785851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:53.786519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.786801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.787087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.787951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] RBAC: Adding query filter params (group.id=32af467cae1d4edf9e86453f0a0cb312, role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=3b7d62044f5549a6b53772308a090b05) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:53.788266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:53.789728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:53.797923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 190/382] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:11:53 2022] GET /identity/v3/role_assignments?group.id=32af467cae1d4edf9e86453f0a0cb312&scope.project.id=3b7d62044f5549a6b53772308a090b05&role.id=e07cc5009f624eb9912a837c934e68aa => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:53.820564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e06721-ce14-47ec-ac32-32210d1612c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:53.821397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e06721-ce14-47ec-ac32-32210d1612c3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:53.821674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e06721-ce14-47ec-ac32-32210d1612c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:53.821946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e06721-ce14-47ec-ac32-32210d1612c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:53.825789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:53.835583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e06721-ce14-47ec-ac32-32210d1612c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-19e06721-ce14-47ec-ac32-32210d1612c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:53.844865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:53.845936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:53.847002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 193/383] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:53.852177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:53.859491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:53.999570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ec8e9-0cff-464f-876f-7841f79e462d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:54.045267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:54.045966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:54.046255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:54.046534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:54.047307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:54.047644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:54.049044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:54.060895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 191/384] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:54.066485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e06721-ce14-47ec-ac32-32210d1612c3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:54.188902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e06721-ce14-47ec-ac32-32210d1612c3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:54.232484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84005424-11f4-4246-a884-66b3eaecef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:54.233612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005424-11f4-4246-a884-66b3eaecef58 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:54.233892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005424-11f4-4246-a884-66b3eaecef58 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:54.234166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005424-11f4-4246-a884-66b3eaecef58 None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:54.235467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84005424-11f4-4246-a884-66b3eaecef58 None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:54.236976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84005424-11f4-4246-a884-66b3eaecef58 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:54.238658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 194/385] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:54 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:54.244133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:54.312410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963ec9-79ea-4663-9ec3-9d7a5e1e761a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:54.312672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963ec9-79ea-4663-9ec3-9d7a5e1e761a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:54.312953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963ec9-79ea-4663-9ec3-9d7a5e1e761a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:54.314349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 195/386] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:54.319493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86d4e0-22e2-4e54-864c-093b6326b631 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:54.319892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86d4e0-22e2-4e54-864c-093b6326b631 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:54.320241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86d4e0-22e2-4e54-864c-093b6326b631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:54.324249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:54.342784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fa86d4e0-22e2-4e54-864c-093b6326b631 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:54.344419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fa86d4e0-22e2-4e54-864c-093b6326b631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:54.367615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b275f-60ce-4e9a-a42a-7de933eb983c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:54.404847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900941fc-2f8c-4fbc-a057-3ff0bed71fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:54.405718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900941fc-2f8c-4fbc-a057-3ff0bed71fed None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:54.406000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900941fc-2f8c-4fbc-a057-3ff0bed71fed None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:54.406277 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900941fc-2f8c-4fbc-a057-3ff0bed71fed None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:54.407518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900941fc-2f8c-4fbc-a057-3ff0bed71fed None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:54.408974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900941fc-2f8c-4fbc-a057-3ff0bed71fed None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:54.411115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 192/387] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:54 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:54.419986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900941fc-2f8c-4fbc-a057-3ff0bed71fed None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:54.429870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:54.440760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 196/388] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:54.521072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86d4e0-22e2-4e54-864c-093b6326b631 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:54.552198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900941fc-2f8c-4fbc-a057-3ff0bed71fed None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:54.588231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd45760-62a3-4f60-899f-a9dd499ed0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:54.589261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd45760-62a3-4f60-899f-a9dd499ed0f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:54.589587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd45760-62a3-4f60-899f-a9dd499ed0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:54.589962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd45760-62a3-4f60-899f-a9dd499ed0f4 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e/groups/95d6275f34fb4208a23e42ed2528cd3e/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:54.592559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd45760-62a3-4f60-899f-a9dd499ed0f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b05c4db805c48ef88067735bdedc47e, group_id=95d6275f34fb4208a23e42ed2528cd3e, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:54.599364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd45760-62a3-4f60-899f-a9dd499ed0f4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:54.600020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:54.614379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 193/389] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 17:11:54 2022] PUT /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e/groups/95d6275f34fb4208a23e42ed2528cd3e/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:11:54.691893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86d4e0-22e2-4e54-864c-093b6326b631 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:54.761992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fa981-7c4e-4424-98df-c82674a88a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:54.762915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fa981-7c4e-4424-98df-c82674a88a15 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:54.763208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fa981-7c4e-4424-98df-c82674a88a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:54.763705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fa981-7c4e-4424-98df-c82674a88a15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:54.785069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fa981-7c4e-4424-98df-c82674a88a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f66fa981-7c4e-4424-98df-c82674a88a15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:54.800744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:54.801715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:54.802528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:54.803161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 197/390] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:11:54.808685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd45760-62a3-4f60-899f-a9dd499ed0f4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:54.941766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd45760-62a3-4f60-899f-a9dd499ed0f4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:54.981827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:54.982876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:54.983656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:54.984127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:54.985109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:54.985609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:54.987177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:54.996997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 194/391] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:55.002563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fa981-7c4e-4424-98df-c82674a88a15 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:55.069092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:55.140015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fa981-7c4e-4424-98df-c82674a88a15 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:55.190771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022694b-b8ba-42be-bfa0-e651d8dac92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:55.191665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022694b-b8ba-42be-bfa0-e651d8dac92e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:55.191997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022694b-b8ba-42be-bfa0-e651d8dac92e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:55.192216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022694b-b8ba-42be-bfa0-e651d8dac92e None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:55.193452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022694b-b8ba-42be-bfa0-e651d8dac92e None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:55.194926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022694b-b8ba-42be-bfa0-e651d8dac92e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:55.196950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 198/392] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:55 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:55.202957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:55.285443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:55.340299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95bcc4-28d7-4463-a232-6b4ac018442f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:55.381003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfc4a7-3305-4845-bb39-ddd495b4fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:55.381884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfc4a7-3305-4845-bb39-ddd495b4fb22 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:55.382163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfc4a7-3305-4845-bb39-ddd495b4fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:55.382438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfc4a7-3305-4845-bb39-ddd495b4fb22 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:55.383700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfc4a7-3305-4845-bb39-ddd495b4fb22 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:55.385715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfc4a7-3305-4845-bb39-ddd495b4fb22 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:55.387892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 195/393] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:55 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:55.393362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022694b-b8ba-42be-bfa0-e651d8dac92e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:55.531813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022694b-b8ba-42be-bfa0-e651d8dac92e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:55.562988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:55.580994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b53d70-8c7e-4813-9794-499aacd64da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:55.581905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b53d70-8c7e-4813-9794-499aacd64da4 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:55.582189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b53d70-8c7e-4813-9794-499aacd64da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:55.582464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b53d70-8c7e-4813-9794-499aacd64da4 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e/groups/95d6275f34fb4208a23e42ed2528cd3e/roles/64cbdf4e69fa400b8b2f8ed13ebe5367` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:55.584756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b53d70-8c7e-4813-9794-499aacd64da4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b05c4db805c48ef88067735bdedc47e, group_id=95d6275f34fb4208a23e42ed2528cd3e, role_id=64cbdf4e69fa400b8b2f8ed13ebe5367)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:55.586192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b53d70-8c7e-4813-9794-499aacd64da4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:55.614796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 199/394] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 17:11:55 2022] PUT /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e/groups/95d6275f34fb4208a23e42ed2528cd3e/roles/64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:56.099469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927b426-2cdb-4f5f-be05-4b8ee8daf1d1 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.099984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927b426-2cdb-4f5f-be05-4b8ee8daf1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.100061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927b426-2cdb-4f5f-be05-4b8ee8daf1d1 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.101640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 196/395] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:56.107178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dda06-aa54-4baf-8fb1-67be5d453a21 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.108017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dda06-aa54-4baf-8fb1-67be5d453a21 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.108743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dda06-aa54-4baf-8fb1-67be5d453a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.125610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ef8dda06-aa54-4baf-8fb1-67be5d453a21 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:56.127109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ef8dda06-aa54-4baf-8fb1-67be5d453a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:56.220120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 200/396] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:56.267185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927b426-2cdb-4f5f-be05-4b8ee8daf1d1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:56.401394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927b426-2cdb-4f5f-be05-4b8ee8daf1d1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:56.427064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:56.438804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b1812-6413-420d-92b6-a16d383287ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:56.440173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1812-6413-420d-92b6-a16d383287ef None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.440458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1812-6413-420d-92b6-a16d383287ef None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.440736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1812-6413-420d-92b6-a16d383287ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.452446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b1812-6413-420d-92b6-a16d383287ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b02b1812-6413-420d-92b6-a16d383287ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:56.460501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:56.461457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:56.462314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 197/397] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:56.470109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dda06-aa54-4baf-8fb1-67be5d453a21 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:56.503308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1d9c8-dedf-4af4-b524-09fbd7564bfe None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.503663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1d9c8-dedf-4af4-b524-09fbd7564bfe None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.503898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1d9c8-dedf-4af4-b524-09fbd7564bfe None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.505530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 198/398] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:56.511129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7c62-d51b-4e12-b8d7-d67cd28c39e0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.511606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7c62-d51b-4e12-b8d7-d67cd28c39e0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.511992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7c62-d51b-4e12-b8d7-d67cd28c39e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.533053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-24cb7c62-d51b-4e12-b8d7-d67cd28c39e0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:56.534365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-24cb7c62-d51b-4e12-b8d7-d67cd28c39e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:56.614464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 199/399] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:56.647941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dda06-aa54-4baf-8fb1-67be5d453a21 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:56.649911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb7c62-d51b-4e12-b8d7-d67cd28c39e0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:56.662380 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:56.695552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:56.696233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.696513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.696785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.697603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:56.698406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:56.699829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:56.730335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 201/400] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:56.736323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:56.827113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb7c62-d51b-4e12-b8d7-d67cd28c39e0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:56.853408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:56.861598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9844-74bd-40f4-9f9d-0984934ced4f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:56.876149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a900f-ef50-4dd9-aea3-a4bad098bfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:56.876961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a900f-ef50-4dd9-aea3-a4bad098bfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.877278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a900f-ef50-4dd9-aea3-a4bad098bfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.877551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a900f-ef50-4dd9-aea3-a4bad098bfa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.888985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a900f-ef50-4dd9-aea3-a4bad098bfa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-423a900f-ef50-4dd9-aea3-a4bad098bfa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:56.897970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:56.899062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:56.900055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 200/401] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:11:56.901514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65186a-697e-44d3-aafa-8dcd6b6050f8 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.901514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65186a-697e-44d3-aafa-8dcd6b6050f8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.901757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65186a-697e-44d3-aafa-8dcd6b6050f8 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.906391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06b68e-29f4-434d-94bc-8d90edb2b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:56.907120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 201/402] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:56.907756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06b68e-29f4-434d-94bc-8d90edb2b5bf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.907919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65186a-697e-44d3-aafa-8dcd6b6050f8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:56.908415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06b68e-29f4-434d-94bc-8d90edb2b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.908834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06b68e-29f4-434d-94bc-8d90edb2b5bf None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.910169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06b68e-29f4-434d-94bc-8d90edb2b5bf None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:56.913026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06b68e-29f4-434d-94bc-8d90edb2b5bf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:56.915099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 202/403] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:56 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:56.916574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268eb8-3352-4297-aee6-8052369c9aec None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:56.917078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268eb8-3352-4297-aee6-8052369c9aec None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:56.917503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268eb8-3352-4297-aee6-8052369c9aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:56.937777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b3268eb8-3352-4297-aee6-8052369c9aec None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:56.939698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b3268eb8-3352-4297-aee6-8052369c9aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:56.985834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:57.029267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 203/404] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:57.030442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3268eb8-3352-4297-aee6-8052369c9aec None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.035358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65186a-697e-44d3-aafa-8dcd6b6050f8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.075635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.076316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.076599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.076870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.077798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:57.078420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:57.080082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:57.090787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 202/405] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:57.091738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.138871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:57.156479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3268eb8-3352-4297-aee6-8052369c9aec None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.199128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee56a6-2d4f-4292-888b-270aa7b23ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.200056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee56a6-2d4f-4292-888b-270aa7b23ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.200338 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee56a6-2d4f-4292-888b-270aa7b23ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.200613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee56a6-2d4f-4292-888b-270aa7b23ff7 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.201821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee56a6-2d4f-4292-888b-270aa7b23ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:57.203384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee56a6-2d4f-4292-888b-270aa7b23ff7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:57.208302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 204/406] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:57 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:57.209887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee56a6-2d4f-4292-888b-270aa7b23ff7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.232872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f3380-4ab5-496d-9a23-0124bed44fed None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.268593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:57.273513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d948fdf-e002-4bf6-a99a-d67fc1133810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.274317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d948fdf-e002-4bf6-a99a-d67fc1133810 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.274694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d948fdf-e002-4bf6-a99a-d67fc1133810 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.274974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d948fdf-e002-4bf6-a99a-d67fc1133810 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.287698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d948fdf-e002-4bf6-a99a-d67fc1133810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2d948fdf-e002-4bf6-a99a-d67fc1133810 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:57.297120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:57.298048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:57.298865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 203/407] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:11:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:57.299564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d948fdf-e002-4bf6-a99a-d67fc1133810 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.332958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee56a6-2d4f-4292-888b-270aa7b23ff7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.368759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea89a0-101c-4ef3-9d4c-351fa61d20ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.370114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea89a0-101c-4ef3-9d4c-351fa61d20ff None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.370598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea89a0-101c-4ef3-9d4c-351fa61d20ff None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.371067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea89a0-101c-4ef3-9d4c-351fa61d20ff None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.372542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea89a0-101c-4ef3-9d4c-351fa61d20ff None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:57.374248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea89a0-101c-4ef3-9d4c-351fa61d20ff None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:57.376139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 205/408] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:57 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:57.381431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea89a0-101c-4ef3-9d4c-351fa61d20ff None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.405431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:57.412795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d948fdf-e002-4bf6-a99a-d67fc1133810 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.450611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eaf6-b717-46aa-9e3b-bc91509d1ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.451536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eaf6-b717-46aa-9e3b-bc91509d1ae3 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.451862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eaf6-b717-46aa-9e3b-bc91509d1ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.452258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eaf6-b717-46aa-9e3b-bc91509d1ae3 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109/groups/95d6275f34fb4208a23e42ed2528cd3e/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.454227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51eaf6-b717-46aa-9e3b-bc91509d1ae3 None admin] RBAC: Authorizing `identity:create_grant(project_id=044d245aa6804c66a019fc432a744109, group_id=95d6275f34fb4208a23e42ed2528cd3e, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:57.455787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51eaf6-b717-46aa-9e3b-bc91509d1ae3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:57.479667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 204/409] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 17:11:57 2022] PUT /identity/v3/projects/044d245aa6804c66a019fc432a744109/groups/95d6275f34fb4208a23e42ed2528cd3e/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:57.481059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eaf6-b717-46aa-9e3b-bc91509d1ae3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.499473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea89a0-101c-4ef3-9d4c-351fa61d20ff None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.511481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:57.612204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.613026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.613381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.613732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.614595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:57.614975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:57.616584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:57.619088 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eaf6-b717-46aa-9e3b-bc91509d1ae3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.635755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 206/410] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:57.641101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.674204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66377f0e-6217-4fe1-acc3-1dccf895cb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.676442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66377f0e-6217-4fe1-acc3-1dccf895cb9b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.676902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66377f0e-6217-4fe1-acc3-1dccf895cb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.678429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66377f0e-6217-4fe1-acc3-1dccf895cb9b None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.681504 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66377f0e-6217-4fe1-acc3-1dccf895cb9b None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:57.685316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66377f0e-6217-4fe1-acc3-1dccf895cb9b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:57.690434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 205/411] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:57 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:57.692085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c09dc-e1ef-45e7-a617-440ad556a293 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.692296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c09dc-e1ef-45e7-a617-440ad556a293 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.692573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c09dc-e1ef-45e7-a617-440ad556a293 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.694188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 206/412] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:11:57.695121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c09dc-e1ef-45e7-a617-440ad556a293 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.738568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:57.787971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd10ed-c447-42b5-93e0-e064cc7c4097 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.828237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27f8b3-fba6-4026-9cc1-545f62eaac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.829266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27f8b3-fba6-4026-9cc1-545f62eaac08 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.829542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27f8b3-fba6-4026-9cc1-545f62eaac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.829885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27f8b3-fba6-4026-9cc1-545f62eaac08 None admin] PATH_INFO: `/v3/groups/32af467cae1d4edf9e86453f0a0cb312` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.831196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27f8b3-fba6-4026-9cc1-545f62eaac08 None admin] RBAC: Authorizing `identity:get_group(group_id=32af467cae1d4edf9e86453f0a0cb312)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:57.832707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27f8b3-fba6-4026-9cc1-545f62eaac08 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:57.834581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 207/413] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:57 2022] GET /identity/v3/groups/32af467cae1d4edf9e86453f0a0cb312 => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:57.835660 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd792-9c6a-4646-91e8-4c907ed77f06 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.836008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd792-9c6a-4646-91e8-4c907ed77f06 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.836289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd792-9c6a-4646-91e8-4c907ed77f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.836816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c09dc-e1ef-45e7-a617-440ad556a293 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:57.853646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-437bd792-9c6a-4646-91e8-4c907ed77f06 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:57.855765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-437bd792-9c6a-4646-91e8-4c907ed77f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:57.881261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd9987-6e2e-4db8-b8d4-3491a6a736d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:57.882542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd9987-6e2e-4db8-b8d4-3491a6a736d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:57.883032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd9987-6e2e-4db8-b8d4-3491a6a736d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:57.883561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd9987-6e2e-4db8-b8d4-3491a6a736d3 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109/groups/95d6275f34fb4208a23e42ed2528cd3e/roles/64cbdf4e69fa400b8b2f8ed13ebe5367` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:57.886462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd9987-6e2e-4db8-b8d4-3491a6a736d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=044d245aa6804c66a019fc432a744109, group_id=95d6275f34fb4208a23e42ed2528cd3e, role_id=64cbdf4e69fa400b8b2f8ed13ebe5367)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:57.888345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:57.888909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd9987-6e2e-4db8-b8d4-3491a6a736d3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:57.908395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 207/414] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 17:11:57 2022] PUT /identity/v3/projects/044d245aa6804c66a019fc432a744109/groups/95d6275f34fb4208a23e42ed2528cd3e/roles/64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:57.909732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd9987-6e2e-4db8-b8d4-3491a6a736d3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:57.930291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 208/415] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:11:57.966170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd792-9c6a-4646-91e8-4c907ed77f06 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:58.016739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:58.036799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd9987-6e2e-4db8-b8d4-3491a6a736d3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:58.076037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5e6ee-0bc8-48a7-ba32-153537b73b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:58.077319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5e6ee-0bc8-48a7-ba32-153537b73b38 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:58.077811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5e6ee-0bc8-48a7-ba32-153537b73b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:58.078286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5e6ee-0bc8-48a7-ba32-153537b73b38 None admin] PATH_INFO: `/v3/projects/3b7d62044f5549a6b53772308a090b05` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:58.079965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5e6ee-0bc8-48a7-ba32-153537b73b38 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7d62044f5549a6b53772308a090b05)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:58.081937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5e6ee-0bc8-48a7-ba32-153537b73b38 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:58.084423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 208/416] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:57 2022] GET /identity/v3/projects/3b7d62044f5549a6b53772308a090b05 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:58.090903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5e6ee-0bc8-48a7-ba32-153537b73b38 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:58.169958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd792-9c6a-4646-91e8-4c907ed77f06 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:58.228091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67554b0d-4a19-4d6e-8e8c-2517d3fde3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:58.229506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67554b0d-4a19-4d6e-8e8c-2517d3fde3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:58.230133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67554b0d-4a19-4d6e-8e8c-2517d3fde3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:58.230611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67554b0d-4a19-4d6e-8e8c-2517d3fde3c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:58.235434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5e6ee-0bc8-48a7-ba32-153537b73b38 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:58.246556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67554b0d-4a19-4d6e-8e8c-2517d3fde3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-67554b0d-4a19-4d6e-8e8c-2517d3fde3c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:58.257263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:58.258198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:11:58.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 209/417] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:11:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:11:58.264965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67554b0d-4a19-4d6e-8e8c-2517d3fde3c9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:58.284080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf1df2-19a5-45d7-adf7-b0a11a2c0540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:58.285316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf1df2-19a5-45d7-adf7-b0a11a2c0540 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:58.285801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf1df2-19a5-45d7-adf7-b0a11a2c0540 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:58.286274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf1df2-19a5-45d7-adf7-b0a11a2c0540 None admin] PATH_INFO: `/v3/projects/3b7d62044f5549a6b53772308a090b05/groups/32af467cae1d4edf9e86453f0a0cb312/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:58.288642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf1df2-19a5-45d7-adf7-b0a11a2c0540 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b7d62044f5549a6b53772308a090b05, group_id=32af467cae1d4edf9e86453f0a0cb312, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:58.290843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf1df2-19a5-45d7-adf7-b0a11a2c0540 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:58.316618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 209/418] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 17:11:58 2022] PUT /identity/v3/projects/3b7d62044f5549a6b53772308a090b05/groups/32af467cae1d4edf9e86453f0a0cb312/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:11:58.332969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:58.398750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6c7e5-1d6c-41f2-a0d2-f2d24436446a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:58.399061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6c7e5-1d6c-41f2-a0d2-f2d24436446a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:58.399312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6c7e5-1d6c-41f2-a0d2-f2d24436446a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:58.401007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 210/419] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:11:58.406593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b44fd0-54fd-42d4-ba68-db5aa2a8e1fd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:58.407149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b44fd0-54fd-42d4-ba68-db5aa2a8e1fd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:58.407655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b44fd0-54fd-42d4-ba68-db5aa2a8e1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:58.426331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-64b44fd0-54fd-42d4-ba68-db5aa2a8e1fd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:11:58.427917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-64b44fd0-54fd-42d4-ba68-db5aa2a8e1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:11:58.429948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67554b0d-4a19-4d6e-8e8c-2517d3fde3c9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:58.438920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:58.495515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:58.496225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:58.496508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:58.496781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:58.497566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:58.497879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:58.499353 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:58.520274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 210/420] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:11:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:58.526769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:58.547457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 211/421] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:11:58.586846 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b44fd0-54fd-42d4-ba68-db5aa2a8e1fd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:58.643275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:58.656132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643dcdb-1ded-434f-aed2-65b4e49690e3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:58.694269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f3906-d801-4f42-856e-75eaa69884fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:58.695252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f3906-d801-4f42-856e-75eaa69884fd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:58.695789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f3906-d801-4f42-856e-75eaa69884fd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:58.696305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f3906-d801-4f42-856e-75eaa69884fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:58.723587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b44fd0-54fd-42d4-ba68-db5aa2a8e1fd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:58.785885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f3906-d801-4f42-856e-75eaa69884fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:58.787692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f3906-d801-4f42-856e-75eaa69884fd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:58.791483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3d484-48ce-4648-8572-a83f999b606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:58.791708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3d484-48ce-4648-8572-a83f999b606d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:58.796153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3d484-48ce-4648-8572-a83f999b606d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:58.796153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3d484-48ce-4648-8572-a83f999b606d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:58.803781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3d484-48ce-4648-8572-a83f999b606d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e5f3d484-48ce-4648-8572-a83f999b606d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:11:58.813011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:11:58.814000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:11:58.814791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 212/422] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:11:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:11:58.820546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3d484-48ce-4648-8572-a83f999b606d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:58.824288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:58.968170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3d484-48ce-4648-8572-a83f999b606d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:59.006039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:59.006719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:59.007003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:59.007277 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:59.008225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:59.008673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:59.010724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:59.019992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 213/423] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:11:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:59.025400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:59.045179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:59.065123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 211/424] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:11:59.068593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f3906-d801-4f42-856e-75eaa69884fd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:59.156263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:59.220210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f3906-d801-4f42-856e-75eaa69884fd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:59.223283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9df06-1f41-4b23-9faa-643f2269f54d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:59.282503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4c6bd-61ee-4102-a399-72b03023e762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:59.283262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4c6bd-61ee-4102-a399-72b03023e762 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:59.283475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4c6bd-61ee-4102-a399-72b03023e762 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:59.283711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4c6bd-61ee-4102-a399-72b03023e762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:59.285997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1fa96-0fb9-44ea-960d-3e4c824b15cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:59.286853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1fa96-0fb9-44ea-960d-3e4c824b15cf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:59.287125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1fa96-0fb9-44ea-960d-3e4c824b15cf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:59.287413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1fa96-0fb9-44ea-960d-3e4c824b15cf None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:59.290093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1fa96-0fb9-44ea-960d-3e4c824b15cf None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:59.292354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1fa96-0fb9-44ea-960d-3e4c824b15cf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:59.294046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 212/425] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:59 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:59.299616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1fa96-0fb9-44ea-960d-3e4c824b15cf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:59.391919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4c6bd-61ee-4102-a399-72b03023e762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:59.393291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4c6bd-61ee-4102-a399-72b03023e762 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:59.409377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:59.461818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:59.504784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 214/426] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 17:11:59.511111 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4c6bd-61ee-4102-a399-72b03023e762 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:59.514942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1fa96-0fb9-44ea-960d-3e4c824b15cf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:59.588560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec930-85ae-42ce-ac5d-b61c0d6dcda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:59.589801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec930-85ae-42ce-ac5d-b61c0d6dcda4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:59.590289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec930-85ae-42ce-ac5d-b61c0d6dcda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:59.590767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec930-85ae-42ce-ac5d-b61c0d6dcda4 None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:59.592270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ec930-85ae-42ce-ac5d-b61c0d6dcda4 None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:59.593915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ec930-85ae-42ce-ac5d-b61c0d6dcda4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:59.596103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 213/427] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:59 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:59.601492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec930-85ae-42ce-ac5d-b61c0d6dcda4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:59.634127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4c6bd-61ee-4102-a399-72b03023e762 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:59.684469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0aeac-cdb7-4971-a245-71d9e28aafc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:59.685942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0aeac-cdb7-4971-a245-71d9e28aafc7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:59.686456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0aeac-cdb7-4971-a245-71d9e28aafc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:59.686918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0aeac-cdb7-4971-a245-71d9e28aafc7 None admin] PATH_INFO: `/v3/projects/7b05c4db805c48ef88067735bdedc47e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:59.688407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0aeac-cdb7-4971-a245-71d9e28aafc7 None admin] RBAC: Authorizing `identity:get_project(project_id=7b05c4db805c48ef88067735bdedc47e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:59.689884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0aeac-cdb7-4971-a245-71d9e28aafc7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:59.691926 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 215/428] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:11:59 2022] GET /identity/v3/projects/7b05c4db805c48ef88067735bdedc47e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:59.697195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0aeac-cdb7-4971-a245-71d9e28aafc7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:11:59.732434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:59.744340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec930-85ae-42ce-ac5d-b61c0d6dcda4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:59.772663 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:11:59.784141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:59.785091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:59.785638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:59.786117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:59.787187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] RBAC: Adding query filter params (group.id=95d6275f34fb4208a23e42ed2528cd3e, role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=7b05c4db805c48ef88067735bdedc47e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:11:59.787756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:59.789392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:59.796354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 214/429] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:11:59 2022] GET /identity/v3/role_assignments?group.id=95d6275f34fb4208a23e42ed2528cd3e&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:11:59.851741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0aeac-cdb7-4971-a245-71d9e28aafc7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:11:59.916058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a7df4-de6c-49cc-9ea8-c6905b157d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:11:59.917842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a7df4-de6c-49cc-9ea8-c6905b157d3d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:11:59.918334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a7df4-de6c-49cc-9ea8-c6905b157d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:11:59.918810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a7df4-de6c-49cc-9ea8-c6905b157d3d None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:11:59.920442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a7df4-de6c-49cc-9ea8-c6905b157d3d None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:11:59.922192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a7df4-de6c-49cc-9ea8-c6905b157d3d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:11:59.924486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 216/430] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:11:59 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:11:59.930165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:00.047368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aec4ea-8cd7-4360-b347-a9c0b5654dc2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:00.089188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:00.090256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:00.090753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:00.092836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f443-f90e-4222-9435-27fd9a75f328 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:00.093361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f443-f90e-4222-9435-27fd9a75f328 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:00.093851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f443-f90e-4222-9435-27fd9a75f328 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:00.095431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 217/431] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:12:00.095629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:00.096812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] RBAC: Adding query filter params (group.id=95d6275f34fb4208a23e42ed2528cd3e, role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=7b05c4db805c48ef88067735bdedc47e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:00.097332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:00.100309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12733346-7387-473d-aa2b-ce097418a2b8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:00.100804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12733346-7387-473d-aa2b-ce097418a2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:00.101281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12733346-7387-473d-aa2b-ce097418a2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:00.102815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:00.115522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 215/432] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:11:59 2022] GET /identity/v3/role_assignments?group.id=95d6275f34fb4208a23e42ed2528cd3e&scope.project.id=7b05c4db805c48ef88067735bdedc47e&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:00.125287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-12733346-7387-473d-aa2b-ce097418a2b8 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:00.126507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-12733346-7387-473d-aa2b-ce097418a2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:00.231031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 218/433] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:00.253760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:00.319798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:00.373123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265cdf-2fee-4183-9f01-77dcf068b1f4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:00.413969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cdd53-5f2e-457e-8a7d-377e939cb4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:00.414778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdd53-5f2e-457e-8a7d-377e939cb4cf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:00.415062 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdd53-5f2e-457e-8a7d-377e939cb4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:00.415357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdd53-5f2e-457e-8a7d-377e939cb4cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:00.428127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cdd53-5f2e-457e-8a7d-377e939cb4cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-af7cdd53-5f2e-457e-8a7d-377e939cb4cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:00.441364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:00.442293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:00.443099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 216/434] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:00.449162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12733346-7387-473d-aa2b-ce097418a2b8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:00.488013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:00.564909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12733346-7387-473d-aa2b-ce097418a2b8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:00.606087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f68b24-32c2-4f9b-9f22-2363f1b8f8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:00.607155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68b24-32c2-4f9b-9f22-2363f1b8f8ae None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:00.607675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68b24-32c2-4f9b-9f22-2363f1b8f8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:00.608157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68b24-32c2-4f9b-9f22-2363f1b8f8ae None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:00.609401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f68b24-32c2-4f9b-9f22-2363f1b8f8ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:00.609939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f68b24-32c2-4f9b-9f22-2363f1b8f8ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:00.611683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f68b24-32c2-4f9b-9f22-2363f1b8f8ae None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:00.622520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 219/435] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:12:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:00.627867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cdd53-5f2e-457e-8a7d-377e939cb4cf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:00.752742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cdd53-5f2e-457e-8a7d-377e939cb4cf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:00.768334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:00.784867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd39d9-f493-440f-8b69-e01fdfe06e5c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:00.785477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd39d9-f493-440f-8b69-e01fdfe06e5c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:00.785958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd39d9-f493-440f-8b69-e01fdfe06e5c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:00.787800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 220/436] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:00.793502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5314e-d156-43f8-ac52-9c9c8e52f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:00.794218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5314e-d156-43f8-ac52-9c9c8e52f198 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:00.794462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593824-fcc8-4513-b85e-ea881a79e377 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:00.794591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5314e-d156-43f8-ac52-9c9c8e52f198 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:00.794751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593824-fcc8-4513-b85e-ea881a79e377 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:00.794846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5314e-d156-43f8-ac52-9c9c8e52f198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:00.795027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593824-fcc8-4513-b85e-ea881a79e377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:00.816449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2a593824-fcc8-4513-b85e-ea881a79e377 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:00.817637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2a593824-fcc8-4513-b85e-ea881a79e377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:00.871040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5314e-d156-43f8-ac52-9c9c8e52f198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:00.872464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5314e-d156-43f8-ac52-9c9c8e52f198 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:00.891782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 221/437] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:00.916747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a593824-fcc8-4513-b85e-ea881a79e377 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:00.949825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:00.981453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 217/438] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:00.987271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5314e-d156-43f8-ac52-9c9c8e52f198 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.002514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.035939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a593824-fcc8-4513-b85e-ea881a79e377 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.072329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e79aae-005e-4343-b404-b2b5fa043364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.073690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79aae-005e-4343-b404-b2b5fa043364 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.073966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79aae-005e-4343-b404-b2b5fa043364 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.074235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79aae-005e-4343-b404-b2b5fa043364 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.084660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e79aae-005e-4343-b404-b2b5fa043364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:01.090634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5314e-d156-43f8-ac52-9c9c8e52f198 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d5e79aae-005e-4343-b404-b2b5fa043364 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:01.092978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:01.094091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:01.094906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 222/439] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:01.099872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e79aae-005e-4343-b404-b2b5fa043364 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.129300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51509aa5-b7ae-4844-a050-2dcd295d573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.130110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51509aa5-b7ae-4844-a050-2dcd295d573d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.130393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51509aa5-b7ae-4844-a050-2dcd295d573d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.130668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51509aa5-b7ae-4844-a050-2dcd295d573d None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.131877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51509aa5-b7ae-4844-a050-2dcd295d573d None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:01.133323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51509aa5-b7ae-4844-a050-2dcd295d573d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:01.135806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 218/440] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:12:00 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:01.136823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c9919-7533-4e44-8d35-8d47bc30f28c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.137109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c9919-7533-4e44-8d35-8d47bc30f28c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.137389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c9919-7533-4e44-8d35-8d47bc30f28c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.138659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 219/441] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:01.141054 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c9919-7533-4e44-8d35-8d47bc30f28c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.191952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.213328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e79aae-005e-4343-b404-b2b5fa043364 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.230303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.248673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed1c12-cb3d-42ed-8f7c-c2ef47098d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.249338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed1c12-cb3d-42ed-8f7c-c2ef47098d31 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.249618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed1c12-cb3d-42ed-8f7c-c2ef47098d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.249894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed1c12-cb3d-42ed-8f7c-c2ef47098d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.250160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c9919-7533-4e44-8d35-8d47bc30f28c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.250654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed1c12-cb3d-42ed-8f7c-c2ef47098d31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:01.250964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed1c12-cb3d-42ed-8f7c-c2ef47098d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:01.252383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed1c12-cb3d-42ed-8f7c-c2ef47098d31 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:01.260255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 223/442] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:01.261627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093993e3-3650-4ae0-9d31-859a364d92f2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.261916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093993e3-3650-4ae0-9d31-859a364d92f2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.262194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093993e3-3650-4ae0-9d31-859a364d92f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.278232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-093993e3-3650-4ae0-9d31-859a364d92f2 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:01.279442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-093993e3-3650-4ae0-9d31-859a364d92f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:01.287277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103062e3-85f2-46c7-bca1-e21ae9281fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.288171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103062e3-85f2-46c7-bca1-e21ae9281fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.288460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103062e3-85f2-46c7-bca1-e21ae9281fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.288738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103062e3-85f2-46c7-bca1-e21ae9281fc9 None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.289914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103062e3-85f2-46c7-bca1-e21ae9281fc9 None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:01.291963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103062e3-85f2-46c7-bca1-e21ae9281fc9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:01.293622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 220/443] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:01.294386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103062e3-85f2-46c7-bca1-e21ae9281fc9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.349161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 224/444] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:01.349993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093993e3-3650-4ae0-9d31-859a364d92f2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.398645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103062e3-85f2-46c7-bca1-e21ae9281fc9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.426363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.432715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129939a-75bd-47ff-b2e0-4176321df70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.433366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129939a-75bd-47ff-b2e0-4176321df70b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.433646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129939a-75bd-47ff-b2e0-4176321df70b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.433929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129939a-75bd-47ff-b2e0-4176321df70b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.452193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093993e3-3650-4ae0-9d31-859a364d92f2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.452700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.485890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.486653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.486848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.487080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.487948 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] RBAC: Adding query filter params (group.id=95d6275f34fb4208a23e42ed2528cd3e, role.id=4ef78f9a0aba4d74b1d65fd0740cdc50, scope.project.id=044d245aa6804c66a019fc432a744109) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:01.488259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:01.489694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:01.495742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 225/445] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/role_assignments?group.id=95d6275f34fb4208a23e42ed2528cd3e&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:01.496522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.515039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129939a-75bd-47ff-b2e0-4176321df70b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:01.516567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129939a-75bd-47ff-b2e0-4176321df70b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:01.612379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857d72e-4add-4f23-b8c7-bb67351b7d20 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.643844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 221/446] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:01.649322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129939a-75bd-47ff-b2e0-4176321df70b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.655984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413465f0-63c0-485b-b09d-6782c01ca88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.656834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413465f0-63c0-485b-b09d-6782c01ca88d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.657119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413465f0-63c0-485b-b09d-6782c01ca88d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.657391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413465f0-63c0-485b-b09d-6782c01ca88d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.665299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.669934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413465f0-63c0-485b-b09d-6782c01ca88d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-413465f0-63c0-485b-b09d-6782c01ca88d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:01.678613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:01.679557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:01.680344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 226/447] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:01.686032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413465f0-63c0-485b-b09d-6782c01ca88d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.706813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.780743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129939a-75bd-47ff-b2e0-4176321df70b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.803466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413465f0-63c0-485b-b09d-6782c01ca88d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.824145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7f34-9725-4247-95b7-f747d11c4842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.825352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7f34-9725-4247-95b7-f747d11c4842 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.825628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7f34-9725-4247-95b7-f747d11c4842 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.825898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7f34-9725-4247-95b7-f747d11c4842 None admin] PATH_INFO: `/v3/projects/044d245aa6804c66a019fc432a744109` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.827375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac7f34-9725-4247-95b7-f747d11c4842 None admin] RBAC: Authorizing `identity:get_project(project_id=044d245aa6804c66a019fc432a744109)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:01.828554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac7f34-9725-4247-95b7-f747d11c4842 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:01.832795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 222/448] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/projects/044d245aa6804c66a019fc432a744109 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:01.836753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7f34-9725-4247-95b7-f747d11c4842 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.844899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.845582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.845864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.846144 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.846910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:01.847223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:01.848865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:01.858795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 227/449] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:01.866863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:01.930739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.951578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7f34-9725-4247-95b7-f747d11c4842 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.968703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:01.970956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be28faf-bb27-4e5b-8f03-13dd6f018e39 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:01.986303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69ba62-6106-4bc1-8444-3e5decdc8aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:01.987161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69ba62-6106-4bc1-8444-3e5decdc8aca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:01.987475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69ba62-6106-4bc1-8444-3e5decdc8aca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:01.987745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69ba62-6106-4bc1-8444-3e5decdc8aca None admin] PATH_INFO: `/v3/groups/95d6275f34fb4208a23e42ed2528cd3e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:01.988925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69ba62-6106-4bc1-8444-3e5decdc8aca None admin] RBAC: Authorizing `identity:get_group(group_id=95d6275f34fb4208a23e42ed2528cd3e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:01.990370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69ba62-6106-4bc1-8444-3e5decdc8aca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:01.992026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 223/450] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/groups/95d6275f34fb4208a23e42ed2528cd3e => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:01.997672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69ba62-6106-4bc1-8444-3e5decdc8aca None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:02.004330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bdf40-6085-49ab-9f78-c4148d0d8de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:02.004981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bdf40-6085-49ab-9f78-c4148d0d8de1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:02.005269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bdf40-6085-49ab-9f78-c4148d0d8de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:02.005547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bdf40-6085-49ab-9f78-c4148d0d8de1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:02.077429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bdf40-6085-49ab-9f78-c4148d0d8de1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:02.078906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bdf40-6085-49ab-9f78-c4148d0d8de1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:02.095937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69ba62-6106-4bc1-8444-3e5decdc8aca None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:02.128768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:02.129489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:02.129779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:02.130055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:02.130880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] RBAC: Adding query filter params (group.id=95d6275f34fb4208a23e42ed2528cd3e, role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=044d245aa6804c66a019fc432a744109) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:02.131192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:02.132640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:02.135732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:02.139250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 224/451] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/role_assignments?group.id=95d6275f34fb4208a23e42ed2528cd3e&scope.project.id=044d245aa6804c66a019fc432a744109&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:02.181174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 228/452] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 17:12:02.187664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:02.241658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:02.293671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5be9-61b6-4900-be52-9bf35d00a53f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:02.328262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef3883-3186-418b-a935-0d31198de01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:02.329129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef3883-3186-418b-a935-0d31198de01a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:02.329409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef3883-3186-418b-a935-0d31198de01a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:02.329687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef3883-3186-418b-a935-0d31198de01a None admin] PATH_INFO: `/v3/projects/3b7d62044f5549a6b53772308a090b05` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:02.330860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef3883-3186-418b-a935-0d31198de01a None admin] RBAC: Authorizing `identity:get_project(project_id=3b7d62044f5549a6b53772308a090b05)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:02.332319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef3883-3186-418b-a935-0d31198de01a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:02.334021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 225/453] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:12:02 2022] GET /identity/v3/projects/3b7d62044f5549a6b53772308a090b05 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:02.339923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bdf40-6085-49ab-9f78-c4148d0d8de1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:02.441386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bdf40-6085-49ab-9f78-c4148d0d8de1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:02.482397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe0577-6bea-48ff-bd0e-173f93b39400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:02.483248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe0577-6bea-48ff-bd0e-173f93b39400 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:02.483813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe0577-6bea-48ff-bd0e-173f93b39400 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:02.484081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe0577-6bea-48ff-bd0e-173f93b39400 None admin] PATH_INFO: `/v3/groups/32af467cae1d4edf9e86453f0a0cb312` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:02.485257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe0577-6bea-48ff-bd0e-173f93b39400 None admin] RBAC: Authorizing `identity:get_group(group_id=32af467cae1d4edf9e86453f0a0cb312)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:02.486685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe0577-6bea-48ff-bd0e-173f93b39400 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:02.488324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 229/454] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 17:12:02 2022] GET /identity/v3/groups/32af467cae1d4edf9e86453f0a0cb312 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:02.494269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef3883-3186-418b-a935-0d31198de01a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:02.590696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef3883-3186-418b-a935-0d31198de01a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:02.607150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:02.623494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d0a74-cefc-4e61-9feb-de73a8c76f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:02.624138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0a74-cefc-4e61-9feb-de73a8c76f50 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:02.624416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0a74-cefc-4e61-9feb-de73a8c76f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:02.624687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0a74-cefc-4e61-9feb-de73a8c76f50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:02.625505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d0a74-cefc-4e61-9feb-de73a8c76f50 None admin] RBAC: Adding query filter params (group.id=32af467cae1d4edf9e86453f0a0cb312, role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=3b7d62044f5549a6b53772308a090b05) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:02.625812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d0a74-cefc-4e61-9feb-de73a8c76f50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:02.627221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d0a74-cefc-4e61-9feb-de73a8c76f50 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:02.632817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 226/455] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 17:12:02 2022] GET /identity/v3/role_assignments?group.id=32af467cae1d4edf9e86453f0a0cb312&scope.project.id=3b7d62044f5549a6b53772308a090b05&role.id=e07cc5009f624eb9912a837c934e68aa => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:05.488400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c32dcc-7413-4551-99bf-576565a26749 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:05.488935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c32dcc-7413-4551-99bf-576565a26749 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:05.488935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c32dcc-7413-4551-99bf-576565a26749 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:05.490854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 230/456] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:05.496794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bab80-2ea0-4b8f-bb89-90a02d30ff3e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:05.497277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bab80-2ea0-4b8f-bb89-90a02d30ff3e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:05.497348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bab80-2ea0-4b8f-bb89-90a02d30ff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:05.530448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8d6bab80-2ea0-4b8f-bb89-90a02d30ff3e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:05.531736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8d6bab80-2ea0-4b8f-bb89-90a02d30ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:05.624029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 227/457] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 28 17:12:05.672800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c32dcc-7413-4551-99bf-576565a26749 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:05.710467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:05.746999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54168d-2712-43c3-9c9a-e6009e866d16 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:05.747621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54168d-2712-43c3-9c9a-e6009e866d16 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:05.748128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54168d-2712-43c3-9c9a-e6009e866d16 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:05.750381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 228/458] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:12:05.756352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd40a-b1ab-4de6-ac50-78337968fb87 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:05.756869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd40a-b1ab-4de6-ac50-78337968fb87 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:05.757689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd40a-b1ab-4de6-ac50-78337968fb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:05.790434 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d14dd40a-b1ab-4de6-ac50-78337968fb87 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:05.792194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d14dd40a-b1ab-4de6-ac50-78337968fb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:05.806385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c32dcc-7413-4551-99bf-576565a26749 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:05.845886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:05.847001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:05.847481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:05.847914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:05.860001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:05.861687 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:05.871410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 229/459] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:05.873244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277de-f606-46a5-9cf2-dfd61db7096d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:05.873727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277de-f606-46a5-9cf2-dfd61db7096d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:05.874194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277de-f606-46a5-9cf2-dfd61db7096d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:05.875966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 230/460] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:05.881405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d94cd7-2f15-4efd-a5ce-d72a578e5866 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:05.881940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d94cd7-2f15-4efd-a5ce-d72a578e5866 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:05.882620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d94cd7-2f15-4efd-a5ce-d72a578e5866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:05.883882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:05.884726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:05.885631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:05.885631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:05.885631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:05.885631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:05.885631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 231/461] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:05.891156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:05.913879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-92d94cd7-2f15-4efd-a5ce-d72a578e5866 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:05.915416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-92d94cd7-2f15-4efd-a5ce-d72a578e5866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:05.953418 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:05.986293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 231/462] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:12:05.987460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d94cd7-2f15-4efd-a5ce-d72a578e5866 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.004127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:06.037176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a010-99b8-4d7f-a966-def3099609ef None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.077274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e416b44-93d2-4b59-ab64-dc276758084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.078266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e416b44-93d2-4b59-ab64-dc276758084e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.078743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e416b44-93d2-4b59-ab64-dc276758084e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.079205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e416b44-93d2-4b59-ab64-dc276758084e None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.079969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e416b44-93d2-4b59-ab64-dc276758084e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:06.080275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e416b44-93d2-4b59-ab64-dc276758084e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.081666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e416b44-93d2-4b59-ab64-dc276758084e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.092034 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d94cd7-2f15-4efd-a5ce-d72a578e5866 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.104595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 232/463] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:06.104595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988c851-6a5b-4488-ad1f-6493111e66a7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.104595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988c851-6a5b-4488-ad1f-6493111e66a7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.104595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988c851-6a5b-4488-ad1f-6493111e66a7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.104595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 233/464] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:06.104595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988c851-6a5b-4488-ad1f-6493111e66a7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.134866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.135860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.136146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.136418 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.147287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.148782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:06.167745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:06.168720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:06.169558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:06.169558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:06.169558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:06.169558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:06.169558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 232/465] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:06.169824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.211280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:06.219904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988c851-6a5b-4488-ad1f-6493111e66a7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.261785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90cbca-780a-430d-858a-356eb74622ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.262956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90cbca-780a-430d-858a-356eb74622ec None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.263404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90cbca-780a-430d-858a-356eb74622ec None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.263804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90cbca-780a-430d-858a-356eb74622ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.272689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:06.278632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90cbca-780a-430d-858a-356eb74622ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.280152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90cbca-780a-430d-858a-356eb74622ec None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2e90cbca-780a-430d-858a-356eb74622ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:06.298393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:06.299316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:06.300202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:06.300202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:06.300202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:06.300202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:06.300202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 234/466] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:06.301338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869315b-eb25-42b0-a94c-7c0ac239cda7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.301338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869315b-eb25-42b0-a94c-7c0ac239cda7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.301498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869315b-eb25-42b0-a94c-7c0ac239cda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.310872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e78b8-234b-440d-bb3a-cf3c55ec4184 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.320903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0869315b-eb25-42b0-a94c-7c0ac239cda7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:06.322101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0869315b-eb25-42b0-a94c-7c0ac239cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:06.346958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0608e8-c5cb-4625-b576-777f663c88b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.347589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0608e8-c5cb-4625-b576-777f663c88b4 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.347870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0608e8-c5cb-4625-b576-777f663c88b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.348316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0608e8-c5cb-4625-b576-777f663c88b4 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.348842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0608e8-c5cb-4625-b576-777f663c88b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.350266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0608e8-c5cb-4625-b576-777f663c88b4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.372076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 233/467] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:12:06 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:06.372941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0608e8-c5cb-4625-b576-777f663c88b4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.423726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 235/468] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:06.424999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869315b-eb25-42b0-a94c-7c0ac239cda7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.489552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:06.520094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0608e8-c5cb-4625-b576-777f663c88b4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.574597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.576103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.576103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.576356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.577930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:06.577930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.579022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.580695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869315b-eb25-42b0-a94c-7c0ac239cda7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.592822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 234/469] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:06.593671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.611019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:06.629353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.630306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.631008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.632261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.633011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:06.633322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.636826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.649428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 236/470] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:06.650259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.736497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaba75-9696-4236-bd12-ba869375ddfc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.775040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.776535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.776836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.777116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.786479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:06.789139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.791884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.794087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdc1e6-b87a-46e6-b3ad-5c87655d8698 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:06.815479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:06.816414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:06.817275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:06.817275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:06.817275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:06.817275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:06.817275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 235/471] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:06.818002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.833652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1c7b-4079-47d8-9d2f-9e9d9eddb921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.834682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1c7b-4079-47d8-9d2f-9e9d9eddb921 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.835168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1c7b-4079-47d8-9d2f-9e9d9eddb921 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.835683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1c7b-4079-47d8-9d2f-9e9d9eddb921 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.836592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1c7b-4079-47d8-9d2f-9e9d9eddb921 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.838355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1c7b-4079-47d8-9d2f-9e9d9eddb921 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.861608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 237/472] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:12:06 2022] POST /identity/v3/users => generated 246 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:06.862913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1c7b-4079-47d8-9d2f-9e9d9eddb921 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:06.912164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:06.938648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab608ba6-d4f8-4fda-b9ec-14d0709e26ab None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:06.978198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f77f8-e63d-4a1c-931e-da5b62712cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:06.978949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f77f8-e63d-4a1c-931e-da5b62712cd6 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:06.979239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f77f8-e63d-4a1c-931e-da5b62712cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:06.979632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f77f8-e63d-4a1c-931e-da5b62712cd6 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:06.980377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f77f8-e63d-4a1c-931e-da5b62712cd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:06.981935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f77f8-e63d-4a1c-931e-da5b62712cd6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:06.993643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1c7b-4079-47d8-9d2f-9e9d9eddb921 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:07.012426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 236/473] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:12:06 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:07.015682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b18ad7-b334-4cdb-bf31-f709268269c3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:07.016321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b18ad7-b334-4cdb-bf31-f709268269c3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:07.016789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b18ad7-b334-4cdb-bf31-f709268269c3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:07.018883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 237/474] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:07.027486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5daee-6a1d-4d4f-a169-ae7a90984b3b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:07.027486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5daee-6a1d-4d4f-a169-ae7a90984b3b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:07.027486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5daee-6a1d-4d4f-a169-ae7a90984b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:07.063449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e2b5daee-6a1d-4d4f-a169-ae7a90984b3b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:07.064687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e2b5daee-6a1d-4d4f-a169-ae7a90984b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:07.076282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:07.078063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:07.078577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:07.079194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:07.080710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:07.081618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:07.084049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:07.101202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 238/475] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:07.109871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:07.138346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:07.167852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 238/476] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:12:07.188603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5daee-6a1d-4d4f-a169-ae7a90984b3b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:07.246206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:07.250352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395d611-b287-45e7-b08c-de2584181fe0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:07.301850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604401a4-0f07-4879-b1ec-c3453f722ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:07.302448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604401a4-0f07-4879-b1ec-c3453f722ad4 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:07.302730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604401a4-0f07-4879-b1ec-c3453f722ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:07.303013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604401a4-0f07-4879-b1ec-c3453f722ad4 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:07.303767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604401a4-0f07-4879-b1ec-c3453f722ad4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:07.305229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604401a4-0f07-4879-b1ec-c3453f722ad4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:07.319956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5daee-6a1d-4d4f-a169-ae7a90984b3b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:07.339938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 239/477] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:12:07 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:07.363581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:07.364336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:07.364629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:07.364910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:07.365713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:07.367099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:07.380834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.sql.core [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 17:12:07.380834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 28 17:12:07.380834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [parameters: {'id': '60286bbd668647d6bffb732d6c4ccc55', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 28 17:12:07.380834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80261) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() May 28 17:12:07.401490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return ref.to_dict() May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 17:12:07.402445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application yield resource May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.rollback() May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.force_reraise() May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise self.value May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application session.commit() May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 17:12:07.403394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._prepare_impl() May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.flush() May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._flush(objects) May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application compat.raise_( May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application flush_context.execute() May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rec.execute(self) May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 17:12:07.404266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application c = connection._execute_20( May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 17:12:07.406914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 17:12:07.408943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [parameters: {'id': '60286bbd668647d6bffb732d6c4ccc55', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:07.410951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 28 17:12:07.413251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:07.421278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 239/478] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 17:12:07 2022] POST /identity/v3/roles => generated 144 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:07.421437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604401a4-0f07-4879-b1ec-c3453f722ad4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:07.587330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604401a4-0f07-4879-b1ec-c3453f722ad4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:07.634196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ee40c-eeeb-494d-931b-4f17ffdb7721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:07.635080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ee40c-eeeb-494d-931b-4f17ffdb7721 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:07.635390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ee40c-eeeb-494d-931b-4f17ffdb7721 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:07.635679 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ee40c-eeeb-494d-931b-4f17ffdb7721 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:07.647431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ee40c-eeeb-494d-931b-4f17ffdb7721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-5e8ee40c-eeeb-494d-931b-4f17ffdb7721 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:07.656274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:07.657205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:07.658000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 240/479] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:07.663132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:07.733910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:07.796505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfcb21-bd18-4a8d-93d8-a8875b13a6bb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:07.835698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff99724-86e2-490d-94b7-9dc19940b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:07.836730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff99724-86e2-490d-94b7-9dc19940b472 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:07.837218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff99724-86e2-490d-94b7-9dc19940b472 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:07.837747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff99724-86e2-490d-94b7-9dc19940b472 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:07.839506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff99724-86e2-490d-94b7-9dc19940b472 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:07.840030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff99724-86e2-490d-94b7-9dc19940b472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:07.841603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff99724-86e2-490d-94b7-9dc19940b472 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:07.851313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 240/480] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:09.248982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d8fa-c901-4cd0-bec6-b7454480ccb9 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:09.250054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d8fa-c901-4cd0-bec6-b7454480ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:09.250535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d8fa-c901-4cd0-bec6-b7454480ccb9 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:09.252406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 241/481] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:09.257259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cab1b4-44c0-49e3-a971-d89a86eba43a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:09.258113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cab1b4-44c0-49e3-a971-d89a86eba43a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:09.258604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cab1b4-44c0-49e3-a971-d89a86eba43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:09.284320 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-66cab1b4-44c0-49e3-a971-d89a86eba43a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:09.285842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-66cab1b4-44c0-49e3-a971-d89a86eba43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:09.373656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 241/482] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:12:09.412565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70d8fa-c901-4cd0-bec6-b7454480ccb9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:09.436185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:09.551377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70d8fa-c901-4cd0-bec6-b7454480ccb9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:09.601126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4232230-5bee-412b-ab08-8a68d5451e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:09.601952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232230-5bee-412b-ab08-8a68d5451e37 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:09.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232230-5bee-412b-ab08-8a68d5451e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:09.602502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232230-5bee-412b-ab08-8a68d5451e37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:09.621068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4232230-5bee-412b-ab08-8a68d5451e37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c4232230-5bee-412b-ab08-8a68d5451e37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:09.628853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:09.629749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:09.630536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 242/483] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:09.638060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cab1b4-44c0-49e3-a971-d89a86eba43a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:09.687382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:09.707024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a34ee9-fc40-4ee8-9b1f-4ba40e75d3b3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:09.707372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a34ee9-fc40-4ee8-9b1f-4ba40e75d3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:09.707598 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a34ee9-fc40-4ee8-9b1f-4ba40e75d3b3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:09.708961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 243/484] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:09.722116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db603d-731f-4361-a216-75d054732934 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:09.722116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db603d-731f-4361-a216-75d054732934 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:09.722261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db603d-731f-4361-a216-75d054732934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:09.764530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e5db603d-731f-4361-a216-75d054732934 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:09.765870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e5db603d-731f-4361-a216-75d054732934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:09.781827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cab1b4-44c0-49e3-a971-d89a86eba43a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:09.854534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af0ef1-bfec-4579-a12b-6fd30f2e18ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:09.855765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0ef1-bfec-4579-a12b-6fd30f2e18ce None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:09.856250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0ef1-bfec-4579-a12b-6fd30f2e18ce None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:09.857146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0ef1-bfec-4579-a12b-6fd30f2e18ce None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:09.858081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af0ef1-bfec-4579-a12b-6fd30f2e18ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:09.858917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af0ef1-bfec-4579-a12b-6fd30f2e18ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:09.861429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af0ef1-bfec-4579-a12b-6fd30f2e18ce None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:09.873995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 242/485] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:09.875775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286d9e-f19f-4e07-8fc4-bd89b3fc1300 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:09.876279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286d9e-f19f-4e07-8fc4-bd89b3fc1300 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:09.876728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286d9e-f19f-4e07-8fc4-bd89b3fc1300 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:09.878163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 243/486] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:09.883378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 244/487] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:12:09.884220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3286d9e-f19f-4e07-8fc4-bd89b3fc1300 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:09.885357 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660fed-6a60-41a8-911d-f94ff5d2b822 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:09.885660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660fed-6a60-41a8-911d-f94ff5d2b822 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:09.885938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660fed-6a60-41a8-911d-f94ff5d2b822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:09.897547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:09.908924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ad660fed-6a60-41a8-911d-f94ff5d2b822 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:09.910099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ad660fed-6a60-41a8-911d-f94ff5d2b822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:09.985114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 245/488] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:09.992072 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad660fed-6a60-41a8-911d-f94ff5d2b822 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.021257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:10.030689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3286d9e-f19f-4e07-8fc4-bd89b3fc1300 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.079396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.081131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.081860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.082384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.105306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:10.106731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:10.124599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad660fed-6a60-41a8-911d-f94ff5d2b822 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:10.137984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:10.139031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:10.140018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:10.140018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:10.140018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:10.140018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:10.141358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 244/489] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:10.142056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.170496 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb972edf-45ff-4a79-aadd-7d468c38a84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.171306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb972edf-45ff-4a79-aadd-7d468c38a84e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.171616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb972edf-45ff-4a79-aadd-7d468c38a84e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.171893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb972edf-45ff-4a79-aadd-7d468c38a84e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.173884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:10.184162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb972edf-45ff-4a79-aadd-7d468c38a84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cb972edf-45ff-4a79-aadd-7d468c38a84e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:10.191773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:10.192817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 246/490] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:10.194245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb972edf-45ff-4a79-aadd-7d468c38a84e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.277788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed92e7-f81b-4401-a92e-628dcfa5ffe4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.300001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb972edf-45ff-4a79-aadd-7d468c38a84e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.319748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433463bd-3090-4613-8133-f854653e806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.321040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433463bd-3090-4613-8133-f854653e806b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.321584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433463bd-3090-4613-8133-f854653e806b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.322115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433463bd-3090-4613-8133-f854653e806b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.335129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6312e6-c51b-41fe-a619-12c9eead6cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.335906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6312e6-c51b-41fe-a619-12c9eead6cee None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.336231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6312e6-c51b-41fe-a619-12c9eead6cee None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.336455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6312e6-c51b-41fe-a619-12c9eead6cee None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.337183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6312e6-c51b-41fe-a619-12c9eead6cee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:10.337488 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6312e6-c51b-41fe-a619-12c9eead6cee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:10.338138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433463bd-3090-4613-8133-f854653e806b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:10.338962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6312e6-c51b-41fe-a619-12c9eead6cee None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-433463bd-3090-4613-8133-f854653e806b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:10.347975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:10.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:10.349926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:10.351427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 245/491] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:10.351427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433463bd-3090-4613-8133-f854653e806b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.360650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 247/492] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:10.360650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e51b86-90f0-4068-a12d-881a67b7d561 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.360650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e51b86-90f0-4068-a12d-881a67b7d561 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.360650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e51b86-90f0-4068-a12d-881a67b7d561 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.360650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 248/493] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:10.361033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e51b86-90f0-4068-a12d-881a67b7d561 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.373360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:10.426423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:10.475703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433463bd-3090-4613-8133-f854653e806b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.480827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e51b86-90f0-4068-a12d-881a67b7d561 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.511205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.511941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.512230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.512534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.513363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:10.513708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:10.515559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:10.517652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b17907-025e-47e6-aab4-2fe7fc785552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.518318 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17907-025e-47e6-aab4-2fe7fc785552 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.518594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17907-025e-47e6-aab4-2fe7fc785552 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.518867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17907-025e-47e6-aab4-2fe7fc785552 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.519640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b17907-025e-47e6-aab4-2fe7fc785552 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:10.519949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b17907-025e-47e6-aab4-2fe7fc785552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:10.521304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b17907-025e-47e6-aab4-2fe7fc785552 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:10.526133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 246/494] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:10.527309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.534389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 249/495] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:10.535919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842184e0-156a-4ce7-a201-552105f5015b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.536362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842184e0-156a-4ce7-a201-552105f5015b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.536791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842184e0-156a-4ce7-a201-552105f5015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.555279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-842184e0-156a-4ce7-a201-552105f5015b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:10.556566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-842184e0-156a-4ce7-a201-552105f5015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:10.615876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:10.634890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 250/496] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:10.639964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842184e0-156a-4ce7-a201-552105f5015b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.661773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55f6e7-4a49-4c6b-a621-49f9fad24c9b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.690330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:10.702013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.702922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.703208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.703511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.711888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:10.714073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:10.735243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:10.736200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:10.737147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 247/497] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:10.737718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.760953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842184e0-156a-4ce7-a201-552105f5015b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.797141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.797921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.798198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.798472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.808439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:10.809873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:10.826992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:10.827881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:10.828735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:10.828735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:10.828735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:10.828735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:10.828735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 251/498] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:10.829044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.841995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3c37-66bb-403f-a390-408e00228e5c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.871472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:10.877572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.878368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.878645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.878923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.890463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:10.891955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:10.909305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:10.910162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:10.911026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:10.911026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:10.911026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:10.911026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:10.911026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 248/499] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:10.912340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:10.937498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:10.942049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b287d2-f2be-430c-88fa-1abc16f41190 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:10.977575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fa537-7b1b-4664-bdd9-1c84d0b93c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:10.978356 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa537-7b1b-4664-bdd9-1c84d0b93c81 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:10.978846 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa537-7b1b-4664-bdd9-1c84d0b93c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:10.979122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa537-7b1b-4664-bdd9-1c84d0b93c81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:10.989407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0fa537-7b1b-4664-bdd9-1c84d0b93c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-4d0fa537-7b1b-4664-bdd9-1c84d0b93c81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:10.997309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:10.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 252/500] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:11.005113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e356fa0-3849-4c66-827b-44e66f79a125 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.006316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e356fa0-3849-4c66-827b-44e66f79a125 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.006316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e356fa0-3849-4c66-827b-44e66f79a125 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.006316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 253/501] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:11.006316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e356fa0-3849-4c66-827b-44e66f79a125 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.026634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1046b1-f9d8-4557-a2da-bac6a45670f0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.059157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.059845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.060127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.060395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.061227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:11.061530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.062949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.099542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 249/502] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 17:12:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:11.100361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.110132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e356fa0-3849-4c66-827b-44e66f79a125 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.142253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:11.143466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.144117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.144441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.144795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.145701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:11.146005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.147408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.159847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 254/503] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:11.160629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.181610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:11.200670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf1af3-6266-473d-a243-a43a671c1d0e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.234392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99607d8a-c811-4a20-803c-2f3276197317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.235061 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607d8a-c811-4a20-803c-2f3276197317 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.235368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607d8a-c811-4a20-803c-2f3276197317 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.235637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607d8a-c811-4a20-803c-2f3276197317 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.236354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99607d8a-c811-4a20-803c-2f3276197317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:11.236652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99607d8a-c811-4a20-803c-2f3276197317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.238006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99607d8a-c811-4a20-803c-2f3276197317 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.250498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 250/504] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:11.251791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2791-66cc-422f-80d1-48a943d2673a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.252078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2791-66cc-422f-80d1-48a943d2673a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.252350 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2791-66cc-422f-80d1-48a943d2673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.267620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951233a-0f70-4c2d-9640-f0db964d0e24 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.269102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-687e2791-66cc-422f-80d1-48a943d2673a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:11.270292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-687e2791-66cc-422f-80d1-48a943d2673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:11.301588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.302305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.302586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.302855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.303629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:11.303938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.305309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.313056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 255/505] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:11.313844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.336605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 251/506] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:11.337434 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e2791-66cc-422f-80d1-48a943d2673a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.367139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:11.398394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:11.417197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e56c9-1f17-4b7d-9a2c-724fe733677b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.436488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e2791-66cc-422f-80d1-48a943d2673a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.451470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001164bf-fcc2-425d-8eb8-5aa7ca825e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.452193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001164bf-fcc2-425d-8eb8-5aa7ca825e83 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.452480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001164bf-fcc2-425d-8eb8-5aa7ca825e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.452763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001164bf-fcc2-425d-8eb8-5aa7ca825e83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.471389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.472173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.472452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.472724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.480351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.481791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:11.497098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:11.498046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:11.498935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 252/507] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:11.499644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.525054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001164bf-fcc2-425d-8eb8-5aa7ca825e83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.526748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001164bf-fcc2-425d-8eb8-5aa7ca825e83 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.591929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:11.601931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa86c3-be45-4d56-9f63-33c4ae3b4593 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.605750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:11.626173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 256/508] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:11.626991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001164bf-fcc2-425d-8eb8-5aa7ca825e83 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.636452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.637243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.637522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.637794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.645248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.646689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:11.661139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:11.662076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 253/509] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:11.662982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.726676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001164bf-fcc2-425d-8eb8-5aa7ca825e83 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.761936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.762718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.763001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.763273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.763734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fa1c7-093a-4dc6-8831-90a984e888f8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.773118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.774570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:11.792467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:11.793304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:11.794159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:11.794159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:11.794159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:11.794159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:11.794159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 257/510] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:11.794872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.800381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.801173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.801457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.801733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.812287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.813747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.828023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:11.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:11.831943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:11.832775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:11.832775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:11.832775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:11.832775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:11.832775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 254/511] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:11.833210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:11.835790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:11.905554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c00ac2-893b-48a7-9784-92e9de48b37a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.938751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.939447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.939727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.939999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.940831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:11.941135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.941677 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c14863-c457-4e84-a072-7f4e74e810b7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:11.942574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.975037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:11.975862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:11.976146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:11.976422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:11.987792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:11.988665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 258/512] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:11.989252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:11.989495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:12.009556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:12.010889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:12.011947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:12.011947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:12.011947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:12.011947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:12.011947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 255/513] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:12.011947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.066950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:12.087302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:12.098196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0c95-8673-4558-9145-fdb1d192b723 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:12.115014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc0278-2ac5-4d43-ae1a-031f7c257ab9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:12.137057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:12.137798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:12.138114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:12.138426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:12.139284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:12.139725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:12.141325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:12.149134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:12.149791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:12.150071 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:12.150392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:12.151115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:12.151452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:12.152816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:12.159841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 259/514] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 17:12:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:12.160492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.170053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 256/515] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:12.170053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.261888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955bd56-ce57-4bf3-9966-5a7be9c8654c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:12.279948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25462a30-5e4f-4d75-8f2b-ce87d12e2e14 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:12.295702 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:12.296686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:12.297790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:12.298183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:12.298557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:12.299287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:12.299630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:12.301024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:12.312640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 260/516] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:12.313440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.317382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc0e9-3261-44e6-8cd9-cd7fc17d5fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:12.318040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc0e9-3261-44e6-8cd9-cd7fc17d5fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:12.318320 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc0e9-3261-44e6-8cd9-cd7fc17d5fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:12.318596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc0e9-3261-44e6-8cd9-cd7fc17d5fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:12.326237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:12.389594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdc0e9-3261-44e6-8cd9-cd7fc17d5fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:12.391047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdc0e9-3261-44e6-8cd9-cd7fc17d5fa3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:12.414453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f02b2a-1f59-4012-b9d9-5f340c155522 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:12.448544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:12.449270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:12.449576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:12.449852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:12.450578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:12.450883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:12.453215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:12.466321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 261/517] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:12.467172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.507515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 257/518] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:12.507515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc0e9-3261-44e6-8cd9-cd7fc17d5fa3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.521059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:12.549391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:12.586151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd808f0-a77d-414f-88ea-1f606fca5b8d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:12.634049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a91322-a6b4-46cb-8bc1-b0739857963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:12.635581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a91322-a6b4-46cb-8bc1-b0739857963a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:12.636045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a91322-a6b4-46cb-8bc1-b0739857963a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:12.636537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a91322-a6b4-46cb-8bc1-b0739857963a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:12.646804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc0e9-3261-44e6-8cd9-cd7fc17d5fa3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:12.692644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:12.693676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:12.693795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:12.694002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:12.702524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:12.703980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:12.715134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a91322-a6b4-46cb-8bc1-b0739857963a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:12.716647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a91322-a6b4-46cb-8bc1-b0739857963a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:12.722298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:12.723404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 17:12:12.724539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:12.727476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 258/519] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:12.728780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.793842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:12.805648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:12.829289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 262/520] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:12.830119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a91322-a6b4-46cb-8bc1-b0739857963a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.830974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389afcbb-b647-481b-abf7-c6ff564391bd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:12.864448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56178ec6-9d4a-410e-8a45-0ffd34874ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:12.865112 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178ec6-9d4a-410e-8a45-0ffd34874ce1 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:12.865394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178ec6-9d4a-410e-8a45-0ffd34874ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:12.865670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178ec6-9d4a-410e-8a45-0ffd34874ce1 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:12.866376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56178ec6-9d4a-410e-8a45-0ffd34874ce1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:12.867868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56178ec6-9d4a-410e-8a45-0ffd34874ce1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:12.889015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 259/521] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:12:12 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:12.889839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56178ec6-9d4a-410e-8a45-0ffd34874ce1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:12.959711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a91322-a6b4-46cb-8bc1-b0739857963a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.008948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56178ec6-9d4a-410e-8a45-0ffd34874ce1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.012123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.013024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.013374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.013757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.021432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.022975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:13.039090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:13.040235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:13.041292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 263/522] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:13.042348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.048196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:13.054831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.055821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.057558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.058323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.072033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.074137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.076470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:13.092224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:13.093101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:13.093950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:13.093950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:13.093950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:13.093950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:13.093950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 260/523] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:13.094208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.165258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637e9f1-32b1-474b-b6a3-9e8738431d8f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.202119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.202787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.203087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.203383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.204236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:13.204554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.206015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.212971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6de2a-3903-49fd-8d17-9e9a6dfb5263 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.224445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 264/524] 10.222.0.33 () {56 vars in 1139 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:13.225261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.255458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.256258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.256542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.256818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.268001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.269444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:13.287384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:13.288295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:13.289133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:13.289133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:13.289133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:13.289133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:13.289133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 261/525] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:13.289422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.319639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:13.335739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f89def-c73a-4bbe-9bc1-9ac1166a3a2f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.335905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:13.372982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.373639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.373911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.374180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.374966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:13.375269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.376759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.400084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 265/526] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:13.400084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.411476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05f99c-7466-48e2-8e79-5ba855b4b02b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.451246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.451842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.452121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.452393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.453119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:13.453425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.454782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.471413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 262/527] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:13.471413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.510854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a2cdc-b972-4bad-a8f9-11c5afef88e1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.546761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea7a4a-8c8d-4543-b54a-7e4ca0bba4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.547627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea7a4a-8c8d-4543-b54a-7e4ca0bba4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.547912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea7a4a-8c8d-4543-b54a-7e4ca0bba4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.548186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea7a4a-8c8d-4543-b54a-7e4ca0bba4e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.564782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:13.574649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:13.582218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1fd58-8d69-4d30-8765-cdf533555150 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.620645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.621311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.621591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.621863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.622581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:13.622885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.624281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.627131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea7a4a-8c8d-4543-b54a-7e4ca0bba4e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.628810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea7a4a-8c8d-4543-b54a-7e4ca0bba4e3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.636464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 263/528] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:13.637264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.748520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 266/529] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:13.748520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea7a4a-8c8d-4543-b54a-7e4ca0bba4e3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.751428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaea2c-4e82-4994-a1c9-7c94317367b5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.788929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.789587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.789873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.790152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.792022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=58b8dd344e8e43ec9cec222737f98ea7) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:13.792410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.794248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.798860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:13.811379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 264/530] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/role_assignments?user.id=58b8dd344e8e43ec9cec222737f98ea7&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:13.811379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.823429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:13.875743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea7a4a-8c8d-4543-b54a-7e4ca0bba4e3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.916914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dac712-b9ed-403f-a3a7-0ad5c30017e7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:13.925195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.926928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.927321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.927735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.939112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.940824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.958007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:13.958842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:13.959161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:13.959506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:13.960222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:13.961163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:13.961997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:13.961997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:13.961997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:13.961997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:13.961997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:13.961997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:13.961997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:13.961997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:13.963487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 267/531] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:13.968381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:13.973926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:13.975563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:13.992937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:13.994159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:13.995191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:13.995191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:13.995191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:13.995191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:13.995191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:13.995191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:13.995191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:13.995191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:13.999716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 265/532] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:14.000781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:14.104326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec06e46-81da-4ccf-8216-66cf76f065d8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:14.109644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:14.112390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:14.141317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e754b-67e1-4dac-a1ae-9f2e7eb6951e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:14.154979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:14.156425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:14.157258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:14.157793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:14.176262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:14.178785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:14.180495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:14.181281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:14.182305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:14.182835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:14.184033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:14.184205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:14.185792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:14.203530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:14.204559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:14.205470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:14.205470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:14.205470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:14.205470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:14.205715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 266/533] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:14.206778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:14.208304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 268/534] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:14.216377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:14.332300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8adf1-bf75-42b4-b2be-a681ad4297f4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:14.332942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3ccf0-191f-4297-8d9b-291ecbb98b6c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:14.370748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:14.371458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:14.371843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:14.372174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:14.373006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:14.373316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:14.374939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:14.378095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:14.378759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:14.379039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:14.379313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:14.380127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:14.380432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:14.382430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:14.388376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 269/535] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:14.389214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:14.396815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:14.404311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 267/536] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:14.405497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:14.413389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:14.526476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfa45c-bdf3-455a-b207-5776e1d6745f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:14.527980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad40c7-94ea-4515-ae86-fa228cc53656 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:14.563901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:14.565014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:14.565602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:14.566147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:14.571509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:14.572228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:14.572558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:14.572893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:14.574075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=5b9888d2731f4117b5fc2c3a720931c4) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:14.574402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:14.575051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:14.575857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:14.576935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:14.591581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 270/537] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:14 2022] GET /identity/v3/role_assignments?user.id=5b9888d2731f4117b5fc2c3a720931c4&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:14.594356 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:14.602231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:14.603184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:14.604720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:14.604720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:14.604720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:14.604720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:14.604720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:14.604720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:14.604720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:14.604720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:14.607624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 268/538] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:14.617984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:14.755640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:14.795002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:14.821113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937464-54b3-4bdf-b7e7-2da954eb3864 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:14.830117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2164b22-c4a7-4e1e-ac2c-3171a193cc64 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:14.859849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:14.860509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:14.860787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:14.861058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:14.861998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=57af250e151241f4afc9b38d51a5fbdc) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:14.862312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:14.863936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:14.867541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35a2d3-9490-4ee7-897f-038ca774ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:14.868208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35a2d3-9490-4ee7-897f-038ca774ef48 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:14.868492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35a2d3-9490-4ee7-897f-038ca774ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:14.868765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35a2d3-9490-4ee7-897f-038ca774ef48 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:14.869870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35a2d3-9490-4ee7-897f-038ca774ef48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:14.870177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35a2d3-9490-4ee7-897f-038ca774ef48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:14.871624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35a2d3-9490-4ee7-897f-038ca774ef48 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:14.875952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 271/539] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:14 2022] GET /identity/v3/role_assignments?user.id=57af250e151241f4afc9b38d51a5fbdc&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:14.888697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 269/540] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:14.894746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:15.067751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca781aff-0f15-4f7a-baa5-69db11181280 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:15.125326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ca255-bf50-4501-b4fa-ea12bd7f1453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:15.126081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca255-bf50-4501-b4fa-ea12bd7f1453 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:15.126388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca255-bf50-4501-b4fa-ea12bd7f1453 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:15.126700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca255-bf50-4501-b4fa-ea12bd7f1453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:15.127663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ca255-bf50-4501-b4fa-ea12bd7f1453 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=71a48b3b4fa94798b158488f08bae743) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:15.128022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ca255-bf50-4501-b4fa-ea12bd7f1453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:15.129730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ca255-bf50-4501-b4fa-ea12bd7f1453 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:15.140463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 272/541] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:14 2022] GET /identity/v3/role_assignments?user.id=71a48b3b4fa94798b158488f08bae743&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:15.267391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd2759-c347-45ad-b4b8-e75b2d7a091c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:15.267740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd2759-c347-45ad-b4b8-e75b2d7a091c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:15.267962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd2759-c347-45ad-b4b8-e75b2d7a091c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:15.269358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 270/542] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:15.276680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4056f5-4894-4084-a58a-55360911f987 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:15.276994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4056f5-4894-4084-a58a-55360911f987 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:15.277296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4056f5-4894-4084-a58a-55360911f987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:15.301422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3c4056f5-4894-4084-a58a-55360911f987 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:15.302750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3c4056f5-4894-4084-a58a-55360911f987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:15.312837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:15.402097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 273/543] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:15.447837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd2759-c347-45ad-b4b8-e75b2d7a091c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:15.561085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd2759-c347-45ad-b4b8-e75b2d7a091c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:15.594622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:15.597616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c593d7a-fb9a-41aa-b20e-94311ba45452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:15.598519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c593d7a-fb9a-41aa-b20e-94311ba45452 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:15.599010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c593d7a-fb9a-41aa-b20e-94311ba45452 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:15.599363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c593d7a-fb9a-41aa-b20e-94311ba45452 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:15.611848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c593d7a-fb9a-41aa-b20e-94311ba45452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8c593d7a-fb9a-41aa-b20e-94311ba45452 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:15.620390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:15.621345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:15.622145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 271/544] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:15.627642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4056f5-4894-4084-a58a-55360911f987 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:15.785498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4056f5-4894-4084-a58a-55360911f987 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:15.839141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:15.840007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:15.840290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:15.840566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:15.841322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:15.841631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:15.843026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:15.855966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 274/545] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:15.860404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c593d7a-fb9a-41aa-b20e-94311ba45452 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:16.002683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c593d7a-fb9a-41aa-b20e-94311ba45452 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:16.039659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:16.050708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:16.050708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:16.050708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:16.055741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:16.057287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:16.082333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:16.084511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:16.091483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:16.091483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:16.091483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:16.091483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:16.091483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 272/546] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:16.092217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:16.092217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:16.254095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884d80-87ce-4c96-9f7f-ecccfaef7321 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:16.300362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624afa88-0b90-4f23-aa03-8c200168a1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:16.301466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afa88-0b90-4f23-aa03-8c200168a1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:16.301961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afa88-0b90-4f23-aa03-8c200168a1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:16.302436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afa88-0b90-4f23-aa03-8c200168a1c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:16.303475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624afa88-0b90-4f23-aa03-8c200168a1c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:16.303991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624afa88-0b90-4f23-aa03-8c200168a1c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:16.305648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624afa88-0b90-4f23-aa03-8c200168a1c1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:16.327001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 275/547] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:16.332265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:16.378377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:16.482430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb419aa-ac82-44a9-a4de-f84aa302e814 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:16.529568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9e126-79de-45c6-8c98-243b84a1ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:16.530349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9e126-79de-45c6-8c98-243b84a1ed7c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:16.530637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9e126-79de-45c6-8c98-243b84a1ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:16.530914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9e126-79de-45c6-8c98-243b84a1ed7c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:16.538733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9e126-79de-45c6-8c98-243b84a1ed7c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:16.540202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9e126-79de-45c6-8c98-243b84a1ed7c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:16.563915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7035-0b92-4161-9af3-64656eab75cb None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:16.564468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7035-0b92-4161-9af3-64656eab75cb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:16.564468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7035-0b92-4161-9af3-64656eab75cb None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-24a9e126-79de-45c6-8c98-243b84a1ed7c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:16.564610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:16.566563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:16.567917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:16.568442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 276/549] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:16.568526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 273/549] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:16.572772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c7035-0b92-4161-9af3-64656eab75cb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:16.577470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6071a4-6274-4ef5-94df-7a571e05752e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:16.577763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6071a4-6274-4ef5-94df-7a571e05752e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:16.578043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6071a4-6274-4ef5-94df-7a571e05752e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:16.597816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-af6071a4-6274-4ef5-94df-7a571e05752e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:16.599288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-af6071a4-6274-4ef5-94df-7a571e05752e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:16.698528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 274/550] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:12:16.698808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:16.741275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6071a4-6274-4ef5-94df-7a571e05752e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:16.749745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c7035-0b92-4161-9af3-64656eab75cb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:16.777774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:16.818558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:16.819158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:16.820575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:16.820852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:16.821701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:16.822012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:16.824435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:16.855073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 277/551] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 17:12:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:16.860677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:16.926049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6071a4-6274-4ef5-94df-7a571e05752e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:16.983477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b179b-f5a7-47b0-b04f-ca90ff4157b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:16.984537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b179b-f5a7-47b0-b04f-ca90ff4157b9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:16.984820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b179b-f5a7-47b0-b04f-ca90ff4157b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:16.985095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b179b-f5a7-47b0-b04f-ca90ff4157b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:16.996191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b179b-f5a7-47b0-b04f-ca90ff4157b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a99b179b-f5a7-47b0-b04f-ca90ff4157b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:17.004577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:17.005676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:17.006679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 275/552] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:17.017109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b179b-f5a7-47b0-b04f-ca90ff4157b9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:17.023626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b8fef-eee2-4753-bda3-49a11f1cbc34 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:17.046871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:17.063166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b2193-6f61-4d37-b42b-bf2623386665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:17.064059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b2193-6f61-4d37-b42b-bf2623386665 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.064346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b2193-6f61-4d37-b42b-bf2623386665 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.064643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b2193-6f61-4d37-b42b-bf2623386665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.140104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b2193-6f61-4d37-b42b-bf2623386665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:17.141503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b2193-6f61-4d37-b42b-bf2623386665 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:17.191549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:17.235786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b179b-f5a7-47b0-b04f-ca90ff4157b9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:17.253498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 278/553] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 17:12:17.263915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b2193-6f61-4d37-b42b-bf2623386665 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:17.302181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:17.302873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.303137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.303469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.303733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:17.304235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:17.304540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:17.305925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:17.325638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 276/554] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:17.328727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:17.435044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b2193-6f61-4d37-b42b-bf2623386665 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:17.485211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148794f3-0bc4-45f3-8454-55ac32376899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:17.486440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148794f3-0bc4-45f3-8454-55ac32376899 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.486985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148794f3-0bc4-45f3-8454-55ac32376899 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.487267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148794f3-0bc4-45f3-8454-55ac32376899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.496145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe45ac2-a2d2-4caa-9427-cef97f741aa3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:17.501419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148794f3-0bc4-45f3-8454-55ac32376899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:17.502897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148794f3-0bc4-45f3-8454-55ac32376899 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-148794f3-0bc4-45f3-8454-55ac32376899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:17.528680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:17.529669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:17.530569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:17.530569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:17.530569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:17.530569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:17.534237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 279/555] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:17.534237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ed857-1079-4319-95ab-c00123f81ba3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.534237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ed857-1079-4319-95ab-c00123f81ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.534237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ed857-1079-4319-95ab-c00123f81ba3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.535410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 280/556] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:17.536899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ed857-1079-4319-95ab-c00123f81ba3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:17.543738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2846aaf-9f24-4d6d-8f2d-2414a224d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:17.544683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2846aaf-9f24-4d6d-8f2d-2414a224d116 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.545024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2846aaf-9f24-4d6d-8f2d-2414a224d116 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.545369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2846aaf-9f24-4d6d-8f2d-2414a224d116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.561593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2846aaf-9f24-4d6d-8f2d-2414a224d116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:17.563202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2846aaf-9f24-4d6d-8f2d-2414a224d116 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:17.566050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a2846aaf-9f24-4d6d-8f2d-2414a224d116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:17.588268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:17.589989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:17.591826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:17.591826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:17.591826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:17.591826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:17.595635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 277/557] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:17.596969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcb909-4bb1-4eea-974d-381466df055a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.597252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcb909-4bb1-4eea-974d-381466df055a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.597530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcb909-4bb1-4eea-974d-381466df055a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.618125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-99dcb909-4bb1-4eea-974d-381466df055a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:17.619947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-99dcb909-4bb1-4eea-974d-381466df055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:17.633011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:17.664829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ed857-1079-4319-95ab-c00123f81ba3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:17.711443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 278/558] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:17.711443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcb909-4bb1-4eea-974d-381466df055a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:17.715849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe1d8a-d0f9-4467-84f1-3b327f6b0ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:17.716831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1d8a-d0f9-4467-84f1-3b327f6b0ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.717319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1d8a-d0f9-4467-84f1-3b327f6b0ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.717796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1d8a-d0f9-4467-84f1-3b327f6b0ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.718740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe1d8a-d0f9-4467-84f1-3b327f6b0ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:17.719270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe1d8a-d0f9-4467-84f1-3b327f6b0ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:17.721135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe1d8a-d0f9-4467-84f1-3b327f6b0ad0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:17.745574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 281/559] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:17.747624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ff0c-8fd4-493b-b10b-bd96039c48bc None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.748131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ff0c-8fd4-493b-b10b-bd96039c48bc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.748621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ff0c-8fd4-493b-b10b-bd96039c48bc None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.750124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 282/560] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:17.751046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8ff0c-8fd4-493b-b10b-bd96039c48bc None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:17.819072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcb909-4bb1-4eea-974d-381466df055a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:17.849188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:17.858414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8ff0c-8fd4-493b-b10b-bd96039c48bc None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:17.862961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:17.863661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.863945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.864221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.864957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:17.865267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:17.866643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:17.883722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 279/561] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:17.883722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:17.890019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:17.895442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2524412-422b-4184-8cf9-ef2b2304d1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:17.896230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2524412-422b-4184-8cf9-ef2b2304d1bb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.896509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2524412-422b-4184-8cf9-ef2b2304d1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.896778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2524412-422b-4184-8cf9-ef2b2304d1bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.904332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2524412-422b-4184-8cf9-ef2b2304d1bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:17.905770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2524412-422b-4184-8cf9-ef2b2304d1bb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b2524412-422b-4184-8cf9-ef2b2304d1bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:17.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:17.921105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:17.921985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 283/562] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:17.922686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07fd9-7baa-4fc6-a964-e9a051fc459f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:17.922830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07fd9-7baa-4fc6-a964-e9a051fc459f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:17.923150 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07fd9-7baa-4fc6-a964-e9a051fc459f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:17.939274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bec07fd9-7baa-4fc6-a964-e9a051fc459f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:17.940489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bec07fd9-7baa-4fc6-a964-e9a051fc459f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:17.984321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23148218-0b01-446f-8c1e-91a0d1078d9e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.005541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 284/563] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:18.006951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7abc14-b9d4-449f-af05-6ef37cfbb949 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.007237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7abc14-b9d4-449f-af05-6ef37cfbb949 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.007564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7abc14-b9d4-449f-af05-6ef37cfbb949 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.010464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 285/564] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:18.010464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7abc14-b9d4-449f-af05-6ef37cfbb949 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.020335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22afe6e-ee1b-41b3-9fb5-29775d646ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.021450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afe6e-ee1b-41b3-9fb5-29775d646ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.021917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afe6e-ee1b-41b3-9fb5-29775d646ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.022384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afe6e-ee1b-41b3-9fb5-29775d646ad9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.033562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22afe6e-ee1b-41b3-9fb5-29775d646ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d22afe6e-ee1b-41b3-9fb5-29775d646ad9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:18.041606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:18.042730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 280/565] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:18.043966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22afe6e-ee1b-41b3-9fb5-29775d646ad9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.088315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:18.124654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7abc14-b9d4-449f-af05-6ef37cfbb949 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.143480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22afe6e-ee1b-41b3-9fb5-29775d646ad9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.148138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:18.161249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6177a88-ac38-4d9f-92bf-23e7b8138ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.162038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6177a88-ac38-4d9f-92bf-23e7b8138ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.162317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6177a88-ac38-4d9f-92bf-23e7b8138ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.162591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6177a88-ac38-4d9f-92bf-23e7b8138ce0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.170545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6177a88-ac38-4d9f-92bf-23e7b8138ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:18.172158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6177a88-ac38-4d9f-92bf-23e7b8138ce0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:18.177025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.177681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.177964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.178248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.179035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:18.179364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:18.180785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c6177a88-ac38-4d9f-92bf-23e7b8138ce0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:18.188448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:18.189609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:18.190578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 286/566] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:18.191192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865e1c7-0372-43d1-b979-5a091638a859 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.191192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865e1c7-0372-43d1-b979-5a091638a859 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.191331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865e1c7-0372-43d1-b979-5a091638a859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.198134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 281/567] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:18.199546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.209323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f865e1c7-0372-43d1-b979-5a091638a859 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:18.210524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f865e1c7-0372-43d1-b979-5a091638a859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:18.277615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 287/568] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:18.278679 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865e1c7-0372-43d1-b979-5a091638a859 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.301637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7f668-2e40-4ac4-9100-442bed99b5ed None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.303984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:18.337419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87a0f5-331a-4ac8-a3ee-b931587e4622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.338755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87a0f5-331a-4ac8-a3ee-b931587e4622 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.339033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87a0f5-331a-4ac8-a3ee-b931587e4622 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.339308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87a0f5-331a-4ac8-a3ee-b931587e4622 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.349751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87a0f5-331a-4ac8-a3ee-b931587e4622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2f87a0f5-331a-4ac8-a3ee-b931587e4622 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:18.358138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:18.359451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:18.360289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 282/569] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:18.360990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87a0f5-331a-4ac8-a3ee-b931587e4622 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.394004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865e1c7-0372-43d1-b979-5a091638a859 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.407412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:18.429549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.430216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.430503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.430782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.431564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:18.431869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:18.433267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:18.442342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 288/570] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:18.443108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.459457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87a0f5-331a-4ac8-a3ee-b931587e4622 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.493718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96993654-1085-4252-9588-344b98338e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.494606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993654-1085-4252-9588-344b98338e8f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.494894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993654-1085-4252-9588-344b98338e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.495174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993654-1085-4252-9588-344b98338e8f None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.496045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96993654-1085-4252-9588-344b98338e8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:18.496360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96993654-1085-4252-9588-344b98338e8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:18.497791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96993654-1085-4252-9588-344b98338e8f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:18.514987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 283/571] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:18.515805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96993654-1085-4252-9588-344b98338e8f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.538686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:18.547850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affcd1b-0dc6-4109-9f8a-4ea47eab698f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.580826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.581476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.581761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.582037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.582865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=c4224f58ecbb423dae16d5b9ae09d5cb) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:18.583286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:18.584637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:18.590180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 289/572] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/role_assignments?user.id=c4224f58ecbb423dae16d5b9ae09d5cb&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:18.590960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.622291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96993654-1085-4252-9588-344b98338e8f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.637718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:18.663302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b64fb-d6e0-44fe-9f22-8d705b1db199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.664138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b64fb-d6e0-44fe-9f22-8d705b1db199 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.664417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b64fb-d6e0-44fe-9f22-8d705b1db199 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.664688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b64fb-d6e0-44fe-9f22-8d705b1db199 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.677639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b64fb-d6e0-44fe-9f22-8d705b1db199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-714b64fb-d6e0-44fe-9f22-8d705b1db199 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:18.686190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:18.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 284/573] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:18.688245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b64fb-d6e0-44fe-9f22-8d705b1db199 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.712106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4c882-b882-468b-94ef-50dd72255ef3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.755937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.756623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.756903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.757176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.757942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:18.758253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:18.759843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:18.768966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 290/574] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:18.769883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.799031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:18.820986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b64fb-d6e0-44fe-9f22-8d705b1db199 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.859421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.860220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.860502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.860773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.873285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:18.875046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:18.885987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:18.889382 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f53580-eb0b-49d2-ba0e-ed8c8d841609 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:18.892092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:18.892960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:18.893872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:18.893872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:18.893872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:18.893872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:18.893872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 285/575] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:18.894145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:18.932875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:18.933682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:18.933966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:18.934240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:18.945579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:18.947103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:18.965592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:18.966735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:18.967703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:18.967703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:18.967703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:18.967703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:18.967703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 291/576] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:18.967703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.005108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4643f-f2db-4c2b-bca3-7236d23ac767 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.042060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.042733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.043015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.043453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.044291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:19.044605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.046003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.047420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:19.055373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 286/577] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:19.056126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.086453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5f7ac-8f4e-497e-bbe9-872cc02dc00c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.122607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.123662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.124112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.124603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.135988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.137787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:19.156562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:19.157652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:19.158752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:19.158752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:19.158752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:19.158752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:19.160300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 292/578] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:19.160405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca36e9-26c7-4898-88c0-5cc7f69f07bd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.161296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.167677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:19.199746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.200776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.201056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.201320 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.202033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:19.202336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.203709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.221610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 287/579] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:19.221610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.270536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a3e3-7830-4638-973c-30d98dd25592 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.282175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:19.309358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.310015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.310269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.310573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.311304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:19.311633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.313004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.329620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 293/580] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:19.329620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.334185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568fdf-7dd6-4227-a72a-6f32be65c4dc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.371948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.372748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.373032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.373310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.383816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.385862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.394046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:19.405333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:19.406269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:19.407241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:19.407241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:19.407241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:19.407241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:19.407241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 288/581] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:19.408006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.438689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bdd96-410e-4562-af56-f7b2c16c0767 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.476524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.477189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.477470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.477745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.478671 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:19.479075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.480492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.496119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 294/582] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:19.496119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.519957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45292371-869b-4251-b6f1-553f7a46ac82 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.530638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:19.559008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.560541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.560782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.561060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.570454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.571924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:19.590269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:19.591458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:19.592619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:19.594551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 289/583] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:19.595505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.602792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16743ba9-68d5-4800-ae2d-a393cf93ee28 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.627731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:19.639841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.640641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.640925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.641201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.648116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.649564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:19.664352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:19.665447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 295/584] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:19.666599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.717925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e55110-8375-4563-bbd4-fe288142dd0a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.755133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.755908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.756230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.756538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.757366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:19.757724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.759259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.777413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 290/585] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:19.777413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.791850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391652e1-a59a-456a-8ada-c9250ba328af None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.804459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:19.833899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.834757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.835149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.835458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.843213 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.844709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:19.861324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:19.862322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:19.863275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 296/586] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:19.863912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.886856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a39061-57f1-4132-be21-c1810b96e13c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:19.913283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:19.922358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:19.923317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:19.923629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:19.924065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:19.925232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:19.925700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:19.927127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:19.952005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 291/587] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:19.952005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:19.969416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f88f68-9b41-453c-91a9-1ba53960db75 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.005116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.005784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.006119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.006397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.007190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:20.007528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.008965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.021032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 297/588] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:20.022093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.053961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:20.069805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14cdbe-b314-4dd9-938f-288541b94c82 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.104743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.105469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.105751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.106031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.114439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.115921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:20.131153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:20.132134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 17:12:20.133161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:20.133705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ef19-6afd-4ca4-80aa-e7e9d0a80273 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.135813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:20.137107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 292/589] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:20.137910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.170730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.171431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.171717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.171992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.172840 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:20.173745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.174585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.193387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 298/590] 10.222.0.33 () {56 vars in 1139 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:20.194014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.252317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f48e88-a058-4725-b6e4-a4d9bebcfb72 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.287637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.288753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.289033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.289307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.300740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.301459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df790fbf-d6e3-4571-ac0e-ab1ecf535beb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.302219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.312337 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:20.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:20.331279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:20.332390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:20.332390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:20.332390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:20.332390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:20.332390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 293/591] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:20.332676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.355161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96136082-dd08-448a-881b-df60962c5f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.356023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96136082-dd08-448a-881b-df60962c5f11 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.356358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96136082-dd08-448a-881b-df60962c5f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.356634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96136082-dd08-448a-881b-df60962c5f11 None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/58b8dd344e8e43ec9cec222737f98ea7/roles/7dd141eb611c4f98a92d16b8b21b7a9d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.374674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96136082-dd08-448a-881b-df60962c5f11 None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=58b8dd344e8e43ec9cec222737f98ea7, role_id=7dd141eb611c4f98a92d16b8b21b7a9d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.376155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96136082-dd08-448a-881b-df60962c5f11 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.390640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 299/592] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:20 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/58b8dd344e8e43ec9cec222737f98ea7/roles/7dd141eb611c4f98a92d16b8b21b7a9d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:12:20.391506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96136082-dd08-448a-881b-df60962c5f11 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.409259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:20.474788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398518c3-31ed-447a-b873-a8cbfed19c11 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.515650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.516737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.517229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.517718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96136082-dd08-448a-881b-df60962c5f11 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.517842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.518928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:20.519508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.521300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.542376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 294/593] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:20.543621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.563973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.564895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.565285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.565563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.577875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.579474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:20.600048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:20.601041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:20.601955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:20.601955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:20.601955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:20.601955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:20.601955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 300/594] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:20.602683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.611492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:20.670114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69132640-83d1-4a61-bee2-87cd9fc55000 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.697975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:20.721090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.723733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.724225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.724737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.726384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:20.726914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.729223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.742030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 295/595] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:20.747270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.749093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664016a9-9a2e-494c-bfa8-5f40b6b26823 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.803007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.803838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.804115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.804381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.817706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.819726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:20.840426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:20.841424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:20.842420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:20.842420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:20.842420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:20.842420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:20.843927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 301/596] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:20.844841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.875297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1861f5-294d-4e73-9476-6436002b02d6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.893225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:20.913035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456084d-372d-4c6a-bac7-924b198d5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:20.913704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456084d-372d-4c6a-bac7-924b198d5f68 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.913982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456084d-372d-4c6a-bac7-924b198d5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.914254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456084d-372d-4c6a-bac7-924b198d5f68 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.914973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456084d-372d-4c6a-bac7-924b198d5f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:20.915276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456084d-372d-4c6a-bac7-924b198d5f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:20.917099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456084d-372d-4c6a-bac7-924b198d5f68 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:20.931077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 296/597] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:20.932563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77dce-ca3d-49d4-8fa8-6ba8047d963b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:20.932849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77dce-ca3d-49d4-8fa8-6ba8047d963b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:20.933122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77dce-ca3d-49d4-8fa8-6ba8047d963b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:20.938555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 297/598] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:20.938555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77dce-ca3d-49d4-8fa8-6ba8047d963b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:20.958266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e871ed8-9faf-4a6f-ae73-bdde8454762b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:20.976645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:21.002302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.003116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.003442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.003720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.011310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.012811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:21.027032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:21.028218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 302/599] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:21.032239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.052931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77dce-ca3d-49d4-8fa8-6ba8047d963b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.090663 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb2758-77b5-4338-83b3-05b5718aab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.091357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb2758-77b5-4338-83b3-05b5718aab56 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.091654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb2758-77b5-4338-83b3-05b5718aab56 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.091926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb2758-77b5-4338-83b3-05b5718aab56 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.092657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb2758-77b5-4338-83b3-05b5718aab56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:21.092970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb2758-77b5-4338-83b3-05b5718aab56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.094335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb2758-77b5-4338-83b3-05b5718aab56 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.106300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 298/600] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:21.107593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c98c56-97ca-4021-9007-fa8403a34598 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.107884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c98c56-97ca-4021-9007-fa8403a34598 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.108158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c98c56-97ca-4021-9007-fa8403a34598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.125036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-73c98c56-97ca-4021-9007-fa8403a34598 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:21.126297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-73c98c56-97ca-4021-9007-fa8403a34598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:21.137927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f633c2-07b3-44cf-8237-a14c5febe5f3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.157856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:21.174856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.176007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.176297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.176734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.184083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.185542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.207362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 299/601] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:21.207573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:21.208553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:21.209435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:21.209435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:21.209435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:21.209435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:21.209435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:21.209435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:21.209435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:21.209435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:21.209861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c98c56-97ca-4021-9007-fa8403a34598 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.209937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 303/602] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:21.209937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.251471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:21.323788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c98c56-97ca-4021-9007-fa8403a34598 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.325965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556e04-b8c6-43df-bf43-19955e7d8529 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.368059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.368743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.369181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.369558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.370360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:21.370677 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.370886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.371880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.372165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.372476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.372891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.381877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.384525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 300/603] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:21.384822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.385366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:21.404331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:21.405319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:21.406215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:21.406215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:21.406215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:21.406215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:21.406215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:21.406215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:21.406215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:21.406215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:21.410179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 304/604] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:21.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.421260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:21.500356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2d49c-37d6-48ec-9b66-f3f4775dcc19 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.512142 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:21.517258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76fbfa-cf71-4b45-b34e-83fca44ab60f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.540777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.541538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.541823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.542106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.542914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:21.543226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.544744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.552531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac1084-b0d9-483c-b6ba-f2587049a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.553382 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1084-b0d9-483c-b6ba-f2587049a843 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.553638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1084-b0d9-483c-b6ba-f2587049a843 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.553911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1084-b0d9-483c-b6ba-f2587049a843 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.563198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 301/605] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:21.563198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.569922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac1084-b0d9-483c-b6ba-f2587049a843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-06ac1084-b0d9-483c-b6ba-f2587049a843 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:21.578401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:21.579328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:21.580211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 305/606] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:21.580942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac1084-b0d9-483c-b6ba-f2587049a843 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.655416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:21.676950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e1f9-e860-49e8-93c9-5e75e56e7291 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.694010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac1084-b0d9-483c-b6ba-f2587049a843 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.716801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd86327-daab-4bd9-8b4d-a01e45cb66da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.717703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd86327-daab-4bd9-8b4d-a01e45cb66da None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.717984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd86327-daab-4bd9-8b4d-a01e45cb66da None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.718258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd86327-daab-4bd9-8b4d-a01e45cb66da None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/5b9888d2731f4117b5fc2c3a720931c4/roles/7dd141eb611c4f98a92d16b8b21b7a9d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.734215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd86327-daab-4bd9-8b4d-a01e45cb66da None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=5b9888d2731f4117b5fc2c3a720931c4, role_id=7dd141eb611c4f98a92d16b8b21b7a9d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.735683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd86327-daab-4bd9-8b4d-a01e45cb66da None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.743355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.744568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.744857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.745135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.745921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:21.746237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.747706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.752570 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 302/607] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:21 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/5b9888d2731f4117b5fc2c3a720931c4/roles/7dd141eb611c4f98a92d16b8b21b7a9d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:12:21.753563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd86327-daab-4bd9-8b4d-a01e45cb66da None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.770745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 306/608] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:21.771748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.793302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:21.901816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985677a-49fb-4021-92a6-1ff4fbffa7a8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.908198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd86327-daab-4bd9-8b4d-a01e45cb66da None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:21.951254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.952330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.952841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.953318 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.953567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1603f-423d-4008-a92d-8901bc6e3f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:21.954532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1603f-423d-4008-a92d-8901bc6e3f3e None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:21.954640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:21.954811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1603f-423d-4008-a92d-8901bc6e3f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:21.955087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1603f-423d-4008-a92d-8901bc6e3f3e None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/71a48b3b4fa94798b158488f08bae743/roles/7dd141eb611c4f98a92d16b8b21b7a9d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:21.955524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.957165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:21.958692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:21.969025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 307/609] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:21.970195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:21.976774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1603f-423d-4008-a92d-8901bc6e3f3e None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=71a48b3b4fa94798b158488f08bae743, role_id=7dd141eb611c4f98a92d16b8b21b7a9d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:21.980580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1603f-423d-4008-a92d-8901bc6e3f3e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:22.000129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 303/610] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:21 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/71a48b3b4fa94798b158488f08bae743/roles/7dd141eb611c4f98a92d16b8b21b7a9d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:12:22.001193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1603f-423d-4008-a92d-8901bc6e3f3e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:22.076998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:22.104419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db857630-ad6c-4166-a5dd-837d09446878 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:22.125682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1603f-423d-4008-a92d-8901bc6e3f3e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:22.152405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948f531-460a-44e5-9b04-5968ad1d7c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:22.153680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f531-460a-44e5-9b04-5968ad1d7c04 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:22.154139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f531-460a-44e5-9b04-5968ad1d7c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:22.154672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f531-460a-44e5-9b04-5968ad1d7c04 None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/57af250e151241f4afc9b38d51a5fbdc/roles/7dd141eb611c4f98a92d16b8b21b7a9d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:22.166931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:22.167773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:22.168121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:22.168466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:22.173700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948f531-460a-44e5-9b04-5968ad1d7c04 None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=57af250e151241f4afc9b38d51a5fbdc, role_id=7dd141eb611c4f98a92d16b8b21b7a9d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:22.175577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948f531-460a-44e5-9b04-5968ad1d7c04 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:22.180702 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:22.182875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:22.193330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 308/611] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:21 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/57af250e151241f4afc9b38d51a5fbdc/roles/7dd141eb611c4f98a92d16b8b21b7a9d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:22.202482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:22.204146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:22.205321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:22.205321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:22.205321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:22.205321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:22.208577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 304/612] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:22.210002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948f531-460a-44e5-9b04-5968ad1d7c04 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:22.334307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948f531-460a-44e5-9b04-5968ad1d7c04 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:22.365988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:22.375370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:22.376069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:22.376950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:22.377235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:22.377985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:22.378601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:22.380051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:22.393917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 309/613] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:22.399580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:22.445176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:22.517731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c2049-8ecb-452f-afb0-62b382fcd8f3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:22.553896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:22.554699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:22.554981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:22.555254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:22.563105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:22.564581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:22.580806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:22.581811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:22.582786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:22.587797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:22.588327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 305/614] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:22.750719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc373cf0-e2d2-4993-8889-34b9367baa33 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:22.805140 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc73bfd-8a21-4cc8-8ca0-4f54994a7e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:22.805817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc73bfd-8a21-4cc8-8ca0-4f54994a7e3b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:22.806101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc73bfd-8a21-4cc8-8ca0-4f54994a7e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:22.806409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc73bfd-8a21-4cc8-8ca0-4f54994a7e3b None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:22.807264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc73bfd-8a21-4cc8-8ca0-4f54994a7e3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:22.807599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc73bfd-8a21-4cc8-8ca0-4f54994a7e3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:22.809050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc73bfd-8a21-4cc8-8ca0-4f54994a7e3b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:22.839726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 310/615] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 17:12:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:22.845003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:22.901751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d5516-fa5f-4ca0-9a49-c1bbacc8df3b None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:22.902112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d5516-fa5f-4ca0-9a49-c1bbacc8df3b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:22.902306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d5516-fa5f-4ca0-9a49-c1bbacc8df3b None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:22.903991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 311/616] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:22.908931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273fce-2d78-4269-aa26-c7cdebcbafea None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:22.909219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273fce-2d78-4269-aa26-c7cdebcbafea None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:22.909499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273fce-2d78-4269-aa26-c7cdebcbafea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:22.938146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f4273fce-2d78-4269-aa26-c7cdebcbafea None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:22.939328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f4273fce-2d78-4269-aa26-c7cdebcbafea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:23.001715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:23.004542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7ff21-cce2-487a-9c84-c970c83cdd3b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:23.014058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:23.054916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:23.055945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:23.056233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:23.056536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:23.068533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:23.069994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:23.077597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 312/617] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:23.102972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:23.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:23.105545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:23.105545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:23.105545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:23.105545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:23.105774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4273fce-2d78-4269-aa26-c7cdebcbafea None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:23.106256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 306/618] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:23.118194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:23.235070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4273fce-2d78-4269-aa26-c7cdebcbafea None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:23.287891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4f622-2f3d-4c6d-81cf-af74019e4406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:23.288631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f622-2f3d-4c6d-81cf-af74019e4406 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:23.288913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f622-2f3d-4c6d-81cf-af74019e4406 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:23.289186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f622-2f3d-4c6d-81cf-af74019e4406 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:23.294994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b714235-9f50-4a11-ade1-0bbc95a79d83 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:23.329265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:23.329932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:23.330213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:23.330487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:23.331214 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:23.331546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:23.332918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:23.335986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4f622-2f3d-4c6d-81cf-af74019e4406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b1b4f622-2f3d-4c6d-81cf-af74019e4406 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:23.343927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:23.344880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:23.345726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 313/619] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:23.346445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 307/620] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:23.355942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4f622-2f3d-4c6d-81cf-af74019e4406 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:23.360348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:23.377027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:23.425571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:23.530486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de0a85-3781-4486-9449-171c6718a44b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:23.547715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4f622-2f3d-4c6d-81cf-af74019e4406 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:23.599241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:23.600073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:23.600351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:23.600623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:23.603244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:23.604063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:23.604346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:23.604621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:23.605379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:23.605686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:23.607074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:23.615962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:23.617387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:23.633964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 314/621] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:23.642559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:23.643484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:23.644379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:23.644379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:23.644379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:23.644379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:23.644379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:23.644379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:23.644379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:23.644379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:23.648290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 308/622] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:23.649743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:23.654561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:23.738318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:23.771383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075be3e-0a36-4cb9-a126-4b1daf0fa027 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:23.806077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:23.806697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:23.806980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:23.807371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:23.808168 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:23.808477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:23.809898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:23.827633 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 309/623] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:23.829512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:23.924805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:23.934558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72cebf-42f2-4857-bd60-9e1a3b448f01 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:23.941202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edfa3c-8c61-40b4-86a4-bd37a487eda3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:23.972270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2960c3-84a6-4b4b-924b-24045606de2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:23.973235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2960c3-84a6-4b4b-924b-24045606de2e None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:23.973526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2960c3-84a6-4b4b-924b-24045606de2e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:23.973808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2960c3-84a6-4b4b-924b-24045606de2e None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/c4224f58ecbb423dae16d5b9ae09d5cb/roles/7dd141eb611c4f98a92d16b8b21b7a9d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:23.977376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:23.998386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2960c3-84a6-4b4b-924b-24045606de2e None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=c4224f58ecbb423dae16d5b9ae09d5cb, role_id=7dd141eb611c4f98a92d16b8b21b7a9d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:23.999890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2960c3-84a6-4b4b-924b-24045606de2e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:24.005763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:24.006655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.006938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.007212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.022557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 310/624] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:23 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/c4224f58ecbb423dae16d5b9ae09d5cb/roles/7dd141eb611c4f98a92d16b8b21b7a9d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:12:24.027893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:24.029361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:24.047968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:24.048936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:24.049830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:24.049830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:24.049830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:24.049830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:24.052294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 315/625] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:24.059572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2960c3-84a6-4b4b-924b-24045606de2e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:24.226636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2960c3-84a6-4b4b-924b-24045606de2e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:24.264006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:24.264759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.265066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.265347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.266074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:24.266382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:24.267781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:24.283453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 311/626] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:24.285889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:24.421663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853db1a2-172a-4d77-b558-379aa61a8181 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:24.474011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203bf8d-ad0b-40a3-b744-0ad6bda93d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:24.474827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203bf8d-ad0b-40a3-b744-0ad6bda93d67 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.475104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203bf8d-ad0b-40a3-b744-0ad6bda93d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.475461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203bf8d-ad0b-40a3-b744-0ad6bda93d67 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.488522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203bf8d-ad0b-40a3-b744-0ad6bda93d67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:24.490058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203bf8d-ad0b-40a3-b744-0ad6bda93d67 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2203bf8d-ad0b-40a3-b744-0ad6bda93d67 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:24.507858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:24.508867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:24.510459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 316/627] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:24.515656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:24.558486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f7204-07cf-42bd-bdfe-865d4aa26cb4 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.558812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f7204-07cf-42bd-bdfe-865d4aa26cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.559034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f7204-07cf-42bd-bdfe-865d4aa26cb4 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.560742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 317/628] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:12:24.568350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabc077-5253-4c14-8100-236cc90c0f02 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.568645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabc077-5253-4c14-8100-236cc90c0f02 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.568917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabc077-5253-4c14-8100-236cc90c0f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.571008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:24.598993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dcabc077-5253-4c14-8100-236cc90c0f02 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:24.601436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dcabc077-5253-4c14-8100-236cc90c0f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:24.612527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:24.670157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7d991-0357-4c62-b6cc-34772ccff921 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:24.720665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 318/629] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:24.747133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b05189-17ef-42b8-b25e-ac9b69a7cff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:24.749017 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabc077-5253-4c14-8100-236cc90c0f02 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:24.752534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b05189-17ef-42b8-b25e-ac9b69a7cff5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.753038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b05189-17ef-42b8-b25e-ac9b69a7cff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.757846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b05189-17ef-42b8-b25e-ac9b69a7cff5 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.761020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b05189-17ef-42b8-b25e-ac9b69a7cff5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:24.762737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b05189-17ef-42b8-b25e-ac9b69a7cff5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:24.766763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b05189-17ef-42b8-b25e-ac9b69a7cff5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:24.793139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 312/630] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 17:12:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:24.793139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06cb1c-3fa5-41b0-97e3-8810668660a4 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.793139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06cb1c-3fa5-41b0-97e3-8810668660a4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.793139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06cb1c-3fa5-41b0-97e3-8810668660a4 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.793139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 313/631] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:24.797845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06cb1c-3fa5-41b0-97e3-8810668660a4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:24.904560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabc077-5253-4c14-8100-236cc90c0f02 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:24.913542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:24.920769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06cb1c-3fa5-41b0-97e3-8810668660a4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:24.953587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3bbcd-e6c7-464a-8d93-8e65bcf454ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:24.955622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3bbcd-e6c7-464a-8d93-8e65bcf454ac None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.956163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3bbcd-e6c7-464a-8d93-8e65bcf454ac None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.956675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3bbcd-e6c7-464a-8d93-8e65bcf454ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.959702 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e55b-88c9-4f5f-a408-b70d9773c3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:24.960615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e55b-88c9-4f5f-a408-b70d9773c3fd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.960980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e55b-88c9-4f5f-a408-b70d9773c3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.961349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e55b-88c9-4f5f-a408-b70d9773c3fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:24.968313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:24.972041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3bbcd-e6c7-464a-8d93-8e65bcf454ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-19a3bbcd-e6c7-464a-8d93-8e65bcf454ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:24.984586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:24.985712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:24.986778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:24.989611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 319/632] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:24.991580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baa32d-bcc4-494a-bb83-e41731c03491 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:24.992147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baa32d-bcc4-494a-bb83-e41731c03491 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:24.992638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baa32d-bcc4-494a-bb83-e41731c03491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.014590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-22baa32d-bcc4-494a-bb83-e41731c03491 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:25.016953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-22baa32d-bcc4-494a-bb83-e41731c03491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:25.044678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e55b-88c9-4f5f-a408-b70d9773c3fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:25.046134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e55b-88c9-4f5f-a408-b70d9773c3fd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:25.101509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 320/633] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:25.102155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baa32d-bcc4-494a-bb83-e41731c03491 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.165312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:25.171735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 314/634] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:25.173352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0a834-6312-463b-9a4b-75d69625be28 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.173636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0a834-6312-463b-9a4b-75d69625be28 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.173906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0a834-6312-463b-9a4b-75d69625be28 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.175323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 315/635] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:25.179975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0a834-6312-463b-9a4b-75d69625be28 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.209397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baa32d-bcc4-494a-bb83-e41731c03491 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.240264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:25.246977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.247685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.247967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.248244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.248999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:25.249307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:25.250681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:25.263369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 321/636] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:25.263369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.290548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0a834-6312-463b-9a4b-75d69625be28 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.326620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c62440-be82-4517-b119-92c199690760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.327524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c62440-be82-4517-b119-92c199690760 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.327804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c62440-be82-4517-b119-92c199690760 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.328075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c62440-be82-4517-b119-92c199690760 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.339526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c62440-be82-4517-b119-92c199690760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-76c62440-be82-4517-b119-92c199690760 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:25.347119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:25.348094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:25.348950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 316/637] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:25.350215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0427f87-a5ee-4cf7-a2da-86b2a5f35fb0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.350215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0427f87-a5ee-4cf7-a2da-86b2a5f35fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.350384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0427f87-a5ee-4cf7-a2da-86b2a5f35fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.368210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d0427f87-a5ee-4cf7-a2da-86b2a5f35fb0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:25.369759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d0427f87-a5ee-4cf7-a2da-86b2a5f35fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:25.380815 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d3937-d5bb-4cda-81f2-3dbb2b852d65 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.416718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.417622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.417902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.418174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.428671 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:25.430108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:25.430464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:25.445053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 317/638] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:25.445880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0427f87-a5ee-4cf7-a2da-86b2a5f35fb0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:25.449411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:25.450326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:25.451251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:25.451251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:25.451251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:25.451251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:25.451251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 322/639] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:25.451889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.506019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:25.566016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0427f87-a5ee-4cf7-a2da-86b2a5f35fb0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.571727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e88e73-3a93-46b3-a0e9-cbca0eb60104 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.604296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.605180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.605464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.605738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.607430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.608105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.608390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.608673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.609677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:25.609990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:25.611396 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:25.618104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:25.620521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 323/640] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:25.620730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:25.621313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:25.643202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:25.644406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:25.645510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:25.645510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:25.645510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:25.645510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:25.646574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 318/641] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:25.647429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.694022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:25.741034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0389f-04ba-499e-8576-b28d8e018db6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.749327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:25.765245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092098e-e80e-4de8-9c88-74dddba7b557 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.778625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.779291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.779605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.779940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.780669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:25.780983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:25.782354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:25.798688 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 324/642] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:25.798688 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.809420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646f05-c343-4bc7-b7f5-c626ca7efae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.810570 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646f05-c343-4bc7-b7f5-c626ca7efae9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.810920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646f05-c343-4bc7-b7f5-c626ca7efae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.811261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646f05-c343-4bc7-b7f5-c626ca7efae9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.822376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646f05-c343-4bc7-b7f5-c626ca7efae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3e646f05-c343-4bc7-b7f5-c626ca7efae9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:25.829990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:25.831187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:25.832322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:25.836049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 319/643] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:25.836707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646f05-c343-4bc7-b7f5-c626ca7efae9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.902753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d49a-bad2-482a-993f-1f3e40373dea None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.938492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.939280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.939596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.939849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.943982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:25.947078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646f05-c343-4bc7-b7f5-c626ca7efae9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:25.950729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:25.952324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:25.970734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:25.972252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:25.973331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:25.973331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:25.973331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:25.973331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:25.973331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 325/644] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:25.973331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:25.988003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:25.989435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:25.989960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:25.990482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:25.991579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:25.991963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:25.993399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:25.995229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:26.011545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 320/645] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:26.011545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.095234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70e273-c463-4c7f-b10a-d0db2e9aee7c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.121003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8aed2-22e1-47a9-81dc-a66f828e7a9b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.133008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.133802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.134084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.134383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.142253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.143739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:26.157866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:26.158869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 326/646] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:26.160297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.164457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.165153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.165433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.166267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.167246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:26.167581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.168982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.177754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 321/647] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:26.180226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.208254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:26.250852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:26.274642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a946d-add6-42e2-b95e-5e26dba17e28 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.292725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6641-d8b5-4acc-97a1-56f2c3c8b69c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.311984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.312650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.312932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.313212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.313938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:26.314247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.315650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.332742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 327/648] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:26.332742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.336448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.337273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.337622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.338004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.345935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.347308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:26.362291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:26.363258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:26.364469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:26.368169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 322/649] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:26.369542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.446317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf5439-c774-46b8-a791-47e398e97e13 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.470062 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:26.487307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.487995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.488275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.488549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.489336 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:26.489645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.489957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02a829-1e17-4a36-8cfb-1cc1598e8bda None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.491133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.498676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:26.510331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 328/650] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:26.510331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.532140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.532945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.533228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.533531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.545408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.546861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:26.564159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:26.565249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:26.566186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:26.566186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:26.566186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:26.566186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:26.566186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 323/651] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:26.566565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.624236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b83ca9-b7f0-4506-a6e4-07cad90192d6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.656900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.657781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.658122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.658401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.665233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c0623-ecac-4d6f-b937-e739ee4677c3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.665651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.667878 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:26.681680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:26.687768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 329/652] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:26.689075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.707120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.707829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.708110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.708384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.709218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:26.709533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.710958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.713440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:26.730770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 324/653] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:26.735752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.742139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:26.784331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf622d-9333-4ec2-84f9-d5e710703906 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.816400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f5390-8719-4f55-a05f-fe6e445ce1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.817048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f5390-8719-4f55-a05f-fe6e445ce1ac None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.817327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f5390-8719-4f55-a05f-fe6e445ce1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.817607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f5390-8719-4f55-a05f-fe6e445ce1ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.818419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f5390-8719-4f55-a05f-fe6e445ce1ac None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=58b8dd344e8e43ec9cec222737f98ea7) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:26.818729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f5390-8719-4f55-a05f-fe6e445ce1ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.820181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f5390-8719-4f55-a05f-fe6e445ce1ac None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.826827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 330/654] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/role_assignments?user.id=58b8dd344e8e43ec9cec222737f98ea7&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:26.828069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d0bb5-e3d4-4e52-a46a-010f62f2aec6 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.828348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d0bb5-e3d4-4e52-a46a-010f62f2aec6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.828624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d0bb5-e3d4-4e52-a46a-010f62f2aec6 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.830043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 331/655] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:26.830563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d0bb5-e3d4-4e52-a46a-010f62f2aec6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.836254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810c1b-51c8-4352-82d3-ec621ad2a9ee None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.878156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:26.878794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:26.879078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:26.879384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:26.880126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:26.880431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:26.881979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:26.895270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 325/656] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:26.896199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:26.957498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:26.961303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d0bb5-e3d4-4e52-a46a-010f62f2aec6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:26.987781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:27.001064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabf29d-56e0-46c9-affd-db36fb16f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.002230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabf29d-56e0-46c9-affd-db36fb16f6ce None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.002519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabf29d-56e0-46c9-affd-db36fb16f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.002792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabf29d-56e0-46c9-affd-db36fb16f6ce None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.003851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabf29d-56e0-46c9-affd-db36fb16f6ce None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:27.004167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabf29d-56e0-46c9-affd-db36fb16f6ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:27.005631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabf29d-56e0-46c9-affd-db36fb16f6ce None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:27.015383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df00bb3-8a84-4ab0-b35a-2f904637d983 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:27.028676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 332/657] 10.222.0.33 () {56 vars in 1139 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:27.029839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592dd4-e8b6-4f38-be36-d0bdaaf7ccee None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.030129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592dd4-e8b6-4f38-be36-d0bdaaf7ccee None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.030413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592dd4-e8b6-4f38-be36-d0bdaaf7ccee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.048145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1e592dd4-e8b6-4f38-be36-d0bdaaf7ccee None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:27.049269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1e592dd4-e8b6-4f38-be36-d0bdaaf7ccee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:27.052458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86f77d-7660-4ff6-927c-696469d40142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.053187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86f77d-7660-4ff6-927c-696469d40142 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.053454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86f77d-7660-4ff6-927c-696469d40142 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.054408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86f77d-7660-4ff6-927c-696469d40142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.126063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 333/658] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:27.127033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e592dd4-e8b6-4f38-be36-d0bdaaf7ccee None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.128534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86f77d-7660-4ff6-927c-696469d40142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:27.129948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86f77d-7660-4ff6-927c-696469d40142 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:27.184564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:27.235443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e592dd4-e8b6-4f38-be36-d0bdaaf7ccee None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:27.241262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 326/659] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:27.241262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86f77d-7660-4ff6-927c-696469d40142 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.253205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:27.280685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.281600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.281884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.282158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.289934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:27.291471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:27.305958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:27.307903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 334/660] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:27.309361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.351258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86f77d-7660-4ff6-927c-696469d40142 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:27.388744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb266a5-7042-4482-b90c-bde7b41eff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.389319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb266a5-7042-4482-b90c-bde7b41eff24 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.391156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb266a5-7042-4482-b90c-bde7b41eff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.391156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb266a5-7042-4482-b90c-bde7b41eff24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.412456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:27.420876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedeac7-3f72-47f9-b744-5ffe692748bc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:27.456392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1edbf-f554-4597-a1cd-185e2c01d683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.457344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1edbf-f554-4597-a1cd-185e2c01d683 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.457698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1edbf-f554-4597-a1cd-185e2c01d683 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.458042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1edbf-f554-4597-a1cd-185e2c01d683 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.466469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb266a5-7042-4482-b90c-bde7b41eff24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:27.467977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb266a5-7042-4482-b90c-bde7b41eff24 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:27.469413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1edbf-f554-4597-a1cd-185e2c01d683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d2c1edbf-f554-4597-a1cd-185e2c01d683 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:27.477328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:27.479328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:27.480522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 335/661] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:27.482775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1edbf-f554-4597-a1cd-185e2c01d683 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.520715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:27.582789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 327/662] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:27.583977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb266a5-7042-4482-b90c-bde7b41eff24 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.608229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1edbf-f554-4597-a1cd-185e2c01d683 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:27.638151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:27.646609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.647625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.648066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.648345 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.659112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:27.660608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:27.678353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:27.679362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:27.680271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:27.680271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:27.680271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:27.680271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:27.680271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 336/663] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:27.681061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.692821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb266a5-7042-4482-b90c-bde7b41eff24 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:27.735034 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.735767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.736051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.736328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.737171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:27.737482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:27.738904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:27.756057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 328/664] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:27.756870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.781332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:27.803819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb930c06-c8e0-4ce8-82a1-a4dd88f6cc8a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:27.840423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.841085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.841362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.841632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.842395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:27.842673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:27.844064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:27.856333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 337/665] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:27.856333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.876159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e487e3-d6c9-4179-9c25-e2578e66bea2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:27.887738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:27.912162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:27.912962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:27.913314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:27.913655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:27.924636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:27.926152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:27.944621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:27.945784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:27.951636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:27.951636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:27.951636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:27.951636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:27.951636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 329/666] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:27.951636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:27.964181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78530ef1-3c07-4b99-90aa-d3eb1ff6f2ae None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.000284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.001000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.001224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.001605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.002447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:28.002740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.004147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.018334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 338/667] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:28.019304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.024479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:28.071729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340dc773-767b-4f41-8bad-02653289c3ab None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.111496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01260fbc-7341-4c42-ac9d-a6dfc1723f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.112147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260fbc-7341-4c42-ac9d-a6dfc1723f12 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.112423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260fbc-7341-4c42-ac9d-a6dfc1723f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.112693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260fbc-7341-4c42-ac9d-a6dfc1723f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.140854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c94bb-6927-46d3-aac9-88221a6c9fe2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.144238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:28.178061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.178785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.179061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.179331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.184359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01260fbc-7341-4c42-ac9d-a6dfc1723f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.185830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01260fbc-7341-4c42-ac9d-a6dfc1723f12 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.193250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.194710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:28.213676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:28.214760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:28.215860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:28.215860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:28.215860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:28.215860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:28.215860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 339/668] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:28.220063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.275797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:28.296866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 330/669] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:28.296866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01260fbc-7341-4c42-ac9d-a6dfc1723f12 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.325838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3749bdc-554d-409d-ba29-d173a93b8568 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.360285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:28.361501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.362320 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.362607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.362941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.363666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:28.363971 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.365343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.377129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 340/670] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:28.378550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.404247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01260fbc-7341-4c42-ac9d-a6dfc1723f12 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.444803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.445662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.446013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.446358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.453543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.455050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:28.469879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:28.471009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:28.472150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:28.472150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:28.472150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:28.472150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:28.472150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:28.472150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:28.472150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:28.472150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:28.473375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 331/671] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:28.474200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.500344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9c892-eb6e-40b1-89b2-6edae72961be None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.516436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:28.535132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.535827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.536131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.536381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.537103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:28.537413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.538787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.551384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 341/672] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:28.552195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.589737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0582a20-2c47-4b0d-acc0-66f57f4e4926 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.614362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:28.628639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.629542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.629892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.630235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.643070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.644869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.659509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66156cad-a55d-4e96-8df7-effa72db6812 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:28.663581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:28.664659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:28.665971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:28.665971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:28.665971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:28.665971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:28.667436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 332/673] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:28.668207 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.703962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.704676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.704953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.705224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.714298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.716778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:28.734931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:28.735943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:28.736850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 342/674] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:28.737334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.779622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:28.780159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae18ae-0fc3-4279-837d-b4bddec2c97f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.814842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95266651-1be1-45cb-a331-084752066a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.815712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95266651-1be1-45cb-a331-084752066a6c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.816096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95266651-1be1-45cb-a331-084752066a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.816377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95266651-1be1-45cb-a331-084752066a6c None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.817165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95266651-1be1-45cb-a331-084752066a6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:28.817475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95266651-1be1-45cb-a331-084752066a6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.818907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95266651-1be1-45cb-a331-084752066a6c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.834625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 333/675] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:28.835675 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95266651-1be1-45cb-a331-084752066a6c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.856092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae653e04-f3d1-4887-bd48-6c515f30ab5e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.876374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:28.893839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.894642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.894924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.895199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.902859 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:28.904340 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:28.919716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:28.920877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:28.922074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:28.923705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 343/676] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:28.925046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:28.952268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95266651-1be1-45cb-a331-084752066a6c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:28.996914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:28.997506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:28.997784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:28.998056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:28.998781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:28.999086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.000495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.017178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 334/677] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:29.017178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.032656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:29.039830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef49c-fac1-4dcf-a70b-1825c39f9f42 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.077162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.077919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.078275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.078621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.079728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:29.080144 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.081809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.095635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 344/678] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:29.096621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.115652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:29.124786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83af660-bd28-4945-89a4-1ae4985d3ffe None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.161521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.162174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.162452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.162723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.163574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=5b9888d2731f4117b5fc2c3a720931c4) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:29.163886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.165290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.171007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 335/679] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/role_assignments?user.id=5b9888d2731f4117b5fc2c3a720931c4&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:29.172019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.205087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6ac27-760b-4df5-a9ec-035bf8e3c236 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.246108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.247813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.248666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.249130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.250989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:29.251682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.253742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.277463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 345/680] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:29.278329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.295585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:29.308232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0db86-fec2-42c8-b52b-bcb76cc221de None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.347370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7280ef-c9a4-412f-9028-9da92adc9186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.348294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7280ef-c9a4-412f-9028-9da92adc9186 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.348645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7280ef-c9a4-412f-9028-9da92adc9186 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.348944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7280ef-c9a4-412f-9028-9da92adc9186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.357422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7280ef-c9a4-412f-9028-9da92adc9186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.358874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7280ef-c9a4-412f-9028-9da92adc9186 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-df7280ef-c9a4-412f-9028-9da92adc9186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:29.378049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:29.379227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:29.380215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 336/681] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:29.381017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb095c-d932-4b9b-9365-2d0286617d90 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.381302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb095c-d932-4b9b-9365-2d0286617d90 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.381578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb095c-d932-4b9b-9365-2d0286617d90 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.382867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 337/682] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:29.388179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb095c-d932-4b9b-9365-2d0286617d90 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.398361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4717a-3ecf-494f-88d0-35639a7eb702 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.409184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:29.444642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.445472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.445853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.446229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.447260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=71a48b3b4fa94798b158488f08bae743) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:29.447979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.449594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.463371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 346/683] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/role_assignments?user.id=71a48b3b4fa94798b158488f08bae743&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:29.463371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.533158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb095c-d932-4b9b-9365-2d0286617d90 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.573117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374871-999c-4c68-a21e-1bcfcda5a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.573475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:29.573799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374871-999c-4c68-a21e-1bcfcda5a6ad None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.574324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374871-999c-4c68-a21e-1bcfcda5a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.574441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374871-999c-4c68-a21e-1bcfcda5a6ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.600027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4406582-5a12-42f8-af3c-0727e85db7e8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.642275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb325c-5e43-4fa0-a2de-4de3adcd2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.644241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb325c-5e43-4fa0-a2de-4de3adcd2d35 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.644241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb325c-5e43-4fa0-a2de-4de3adcd2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.644241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb325c-5e43-4fa0-a2de-4de3adcd2d35 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.644839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb325c-5e43-4fa0-a2de-4de3adcd2d35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:29.645239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb325c-5e43-4fa0-a2de-4de3adcd2d35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.646796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb325c-5e43-4fa0-a2de-4de3adcd2d35 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.652139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374871-999c-4c68-a21e-1bcfcda5a6ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.654391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374871-999c-4c68-a21e-1bcfcda5a6ad None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.661188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 347/684] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:29.662591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e807d5-bf0a-4309-8583-e470e643b6c2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.663045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e807d5-bf0a-4309-8583-e470e643b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.667470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e807d5-bf0a-4309-8583-e470e643b6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.681613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:29.689711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-41e807d5-bf0a-4309-8583-e470e643b6c2 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:29.690908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-41e807d5-bf0a-4309-8583-e470e643b6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:29.778223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 338/685] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:29.778987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374871-999c-4c68-a21e-1bcfcda5a6ad None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.785601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 348/686] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:29.786742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e807d5-bf0a-4309-8583-e470e643b6c2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.869800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:29.888807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374871-999c-4c68-a21e-1bcfcda5a6ad None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.911063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e807d5-bf0a-4309-8583-e470e643b6c2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:29.930599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.931885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.932189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.932471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.933598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=57af250e151241f4afc9b38d51a5fbdc) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:29.933913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:29.935549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:29.940150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:29.947405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 339/687] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/role_assignments?user.id=57af250e151241f4afc9b38d51a5fbdc&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:29.947405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:29.973310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:29.974502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:29.974936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:29.975437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:29.999097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:30.002235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:30.042127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:30.043701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:30.044615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:30.044615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:30.044615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:30.044615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:30.052087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 349/688] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:30.058394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:30.129670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334df7cc-5eb0-42c3-a8d0-57897f744231 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:30.167481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:30.168386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:30.168723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:30.169061 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:30.180981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:30.183012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:30.202912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:30.204502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:30.205627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:30.205627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:30.205627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:30.205627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:30.205627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 340/689] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:30.210938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:30.215195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e547fe-8c80-45c4-b736-a76ce5f61b66 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:30.237765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:30.251867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:30.256965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:30.257751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:30.258138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:30.258519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:30.259551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:30.259997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:30.261635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:30.273967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 350/690] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:30.279731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:30.327688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0151e-ce28-4883-87d4-345e1be09a97 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:30.364547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:30.365460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:30.365887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:30.366408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:30.367383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:30.367909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:30.370149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:30.383834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 341/691] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:30.390581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:30.395167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206b049-5472-4704-a018-6d4dbcbc17c0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:30.444723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:30.445603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:30.445891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:30.446168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:30.456555 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:30.459530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:30.477735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:30.478969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:30.479906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 351/692] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:30.485297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:30.514319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:30.524768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9dcf-2eb1-4020-a1ca-3f78b4aeaf43 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:30.580738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:30.616525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10774ed7-4676-4bf4-b1df-af3edc63d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:30.617213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774ed7-4676-4bf4-b1df-af3edc63d4be None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:30.617490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774ed7-4676-4bf4-b1df-af3edc63d4be None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:30.617761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774ed7-4676-4bf4-b1df-af3edc63d4be None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:30.618474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774ed7-4676-4bf4-b1df-af3edc63d4be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:30.624232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774ed7-4676-4bf4-b1df-af3edc63d4be None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:30.649321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586376fa-a31d-4f4f-9f81-09c03e92f950 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:30.667440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 342/693] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:12:30 2022] POST /identity/v3/users => generated 254 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:30.691492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f771d56-a5f8-46e6-98b3-68bd1e113e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:30.692577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f771d56-a5f8-46e6-98b3-68bd1e113e1e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:30.693131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f771d56-a5f8-46e6-98b3-68bd1e113e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:30.693641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f771d56-a5f8-46e6-98b3-68bd1e113e1e None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:30.694713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f771d56-a5f8-46e6-98b3-68bd1e113e1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:30.695286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f771d56-a5f8-46e6-98b3-68bd1e113e1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:30.697640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f771d56-a5f8-46e6-98b3-68bd1e113e1e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:30.711411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 352/694] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:30.718531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10774ed7-4676-4bf4-b1df-af3edc63d4be None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:30.878689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10774ed7-4676-4bf4-b1df-af3edc63d4be None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:31.059264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bdcc1-75b4-4c94-86da-2ff65cf66d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:31.060314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bdcc1-75b4-4c94-86da-2ff65cf66d24 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:31.060822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bdcc1-75b4-4c94-86da-2ff65cf66d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:31.061410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bdcc1-75b4-4c94-86da-2ff65cf66d24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:31.062512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bdcc1-75b4-4c94-86da-2ff65cf66d24 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=c4224f58ecbb423dae16d5b9ae09d5cb) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:31.063021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bdcc1-75b4-4c94-86da-2ff65cf66d24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:31.064663 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bdcc1-75b4-4c94-86da-2ff65cf66d24 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:31.074887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 343/695] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:30 2022] GET /identity/v3/role_assignments?user.id=c4224f58ecbb423dae16d5b9ae09d5cb&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 608 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:31.669501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ccd9-7cc9-43f1-a574-2716edb8ee13 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:31.670069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ccd9-7cc9-43f1-a574-2716edb8ee13 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:31.670158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ccd9-7cc9-43f1-a574-2716edb8ee13 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:31.671812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 353/696] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:31.677594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16daab8-5f01-4c43-a8f3-7357627b1c25 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:31.678025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16daab8-5f01-4c43-a8f3-7357627b1c25 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:31.678205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16daab8-5f01-4c43-a8f3-7357627b1c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:31.696018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f16daab8-5f01-4c43-a8f3-7357627b1c25 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:31.697794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f16daab8-5f01-4c43-a8f3-7357627b1c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:31.708176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:31.775354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 344/697] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 17:12:31.799733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970ccd9-7cc9-43f1-a574-2716edb8ee13 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:31.913811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:31.918875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970ccd9-7cc9-43f1-a574-2716edb8ee13 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:31.955928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e46237-d86b-47ee-84fa-4f30b1bb18cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:31.956867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e46237-d86b-47ee-84fa-4f30b1bb18cf None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:31.957192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e46237-d86b-47ee-84fa-4f30b1bb18cf None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:31.957504 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e46237-d86b-47ee-84fa-4f30b1bb18cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:31.968614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e46237-d86b-47ee-84fa-4f30b1bb18cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-95e46237-d86b-47ee-84fa-4f30b1bb18cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:31.976768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:31.977724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:31.979296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 354/698] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:31.985173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16daab8-5f01-4c43-a8f3-7357627b1c25 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:32.086583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c3b7-ed40-4254-b9e7-bd5c778b306a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.086924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c3b7-ed40-4254-b9e7-bd5c778b306a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.087214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c3b7-ed40-4254-b9e7-bd5c778b306a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.088818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 355/699] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:12:32.094551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10f70a-6762-40a9-af6f-667b25fe9935 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.094881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10f70a-6762-40a9-af6f-667b25fe9935 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.095196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10f70a-6762-40a9-af6f-667b25fe9935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.097485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16daab8-5f01-4c43-a8f3-7357627b1c25 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:32.134069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:32.135450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.135738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.136018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.136441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8a10f70a-6762-40a9-af6f-667b25fe9935 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:32.136782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:32.137092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:32.137712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8a10f70a-6762-40a9-af6f-667b25fe9935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:32.138457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:32.147670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 345/700] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:32.153784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:32.163095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:32.262208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce79c33-9413-4146-92b4-68a283749a11 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:32.297654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:32.298503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.298793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.299073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.310272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 356/701] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:32.311315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:32.313375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:32.332637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:32.333590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:32.334489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:32.334489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:32.334489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:32.334489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:32.334489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 346/702] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:32.339941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10f70a-6762-40a9-af6f-667b25fe9935 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:32.347595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:32.437978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:32.480072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7811e8-0b95-48ce-8a89-262ac88f4880 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:32.529975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:32.530941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.531225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.531763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.532498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:32.534238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:32.549895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:32.551032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:32.552058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:32.552836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 347/703] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:32 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:32.559310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:32.608299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:32.624076 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10f70a-6762-40a9-af6f-667b25fe9935 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:32.680620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:32.681361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.681689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.682007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.682780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:32.683128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:32.684571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:32.701017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dfa09-2188-4fe8-8643-7edc1f7f5ef5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:32.718107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 357/704] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:32.723085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:32.755713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:32.768587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c9eb0-eaaf-494a-a4bc-5c749d3a9e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:32.770383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9eb0-eaaf-494a-a4bc-5c749d3a9e54 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.770663 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9eb0-eaaf-494a-a4bc-5c749d3a9e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.770937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9eb0-eaaf-494a-a4bc-5c749d3a9e54 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.771766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c9eb0-eaaf-494a-a4bc-5c749d3a9e54 None admin] RBAC: Adding query filter params (name=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:32.772126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c9eb0-eaaf-494a-a4bc-5c749d3a9e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:32.773793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c9eb0-eaaf-494a-a4bc-5c749d3a9e54 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:32.780505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 348/705] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:12:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:32.783557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58f5b-3823-471d-bf65-16573555bd6a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.783964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58f5b-3823-471d-bf65-16573555bd6a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.784251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58f5b-3823-471d-bf65-16573555bd6a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.786325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 349/706] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:32.787053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c58f5b-3823-471d-bf65-16573555bd6a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:32.864219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43eda0-6b35-4297-9726-6d6171c6bc51 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:32.900775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173dca3-c1bf-4213-956b-16d8363dce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:32.901613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173dca3-c1bf-4213-956b-16d8363dce0b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.901924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173dca3-c1bf-4213-956b-16d8363dce0b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.902231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173dca3-c1bf-4213-956b-16d8363dce0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.910309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173dca3-c1bf-4213-956b-16d8363dce0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:32.911945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c58f5b-3823-471d-bf65-16573555bd6a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:32.912136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173dca3-c1bf-4213-956b-16d8363dce0b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9173dca3-c1bf-4213-956b-16d8363dce0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:32.927201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:32.928204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:32.929177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 358/707] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:12:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:32.930243 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e76bc-8fc2-46d9-9442-f37baceb7fd1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.930408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e76bc-8fc2-46d9-9442-f37baceb7fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.934648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e76bc-8fc2-46d9-9442-f37baceb7fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.948482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cb0e76bc-8fc2-46d9-9442-f37baceb7fd1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:32.949786 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cb0e76bc-8fc2-46d9-9442-f37baceb7fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:32.953712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:32.954641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:32.955776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:32.956733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:32.956733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:32.957154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:32.957520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:32.959309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:32.971386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 350/708] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:12:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:32.971386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:33.087823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:33.110914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 359/709] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:33.148005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e76bc-8fc2-46d9-9442-f37baceb7fd1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:33.178198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537d68c-49c1-4602-9655-7225afb09de1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:33.230427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:33.231527 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:33.232013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:33.232479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:33.233511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:33.234815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:33.236509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:33.255921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 351/710] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:12:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:33.261440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:33.302938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e76bc-8fc2-46d9-9442-f37baceb7fd1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:33.343979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:33.344719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:33.344997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:33.345271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:33.345934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:33.347325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:33.364693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:33.365677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:33.366502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 360/711] 10.222.0.33 () {56 vars in 1129 bytes} [Sat May 28 17:12:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:33.376288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:33.380164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:33.418154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1751e3e-4abe-412f-9e2b-2ee4a324d0d3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:33.427724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:33.455064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4649b2-4793-46f7-af7a-9e1242ef75a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:33.456091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4649b2-4793-46f7-af7a-9e1242ef75a6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:33.456566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4649b2-4793-46f7-af7a-9e1242ef75a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:33.457031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4649b2-4793-46f7-af7a-9e1242ef75a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:33.536355 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4649b2-4793-46f7-af7a-9e1242ef75a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:33.538199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4649b2-4793-46f7-af7a-9e1242ef75a6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:33.622596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0c122-6be0-4a39-b9d2-5ad29f10b9db None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:33.645229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 352/712] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:33.656687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4649b2-4793-46f7-af7a-9e1242ef75a6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:33.668237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:33.676657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5117b-d5a2-44d5-a357-0bc31e0c6879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:33.677340 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5117b-d5a2-44d5-a357-0bc31e0c6879 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:33.677621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5117b-d5a2-44d5-a357-0bc31e0c6879 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:33.677894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5117b-d5a2-44d5-a357-0bc31e0c6879 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:33.678767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5117b-d5a2-44d5-a357-0bc31e0c6879 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:33.678912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5117b-d5a2-44d5-a357-0bc31e0c6879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:33.680306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5117b-d5a2-44d5-a357-0bc31e0c6879 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:33.686572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 361/713] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 17:12:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:33.687950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fb420-2f11-4721-9021-cb383643f82e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:33.688163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fb420-2f11-4721-9021-cb383643f82e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:33.688439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fb420-2f11-4721-9021-cb383643f82e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:33.689914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 362/714] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:33.696240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a1216-ee71-416f-857e-97dc24c49a0a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:33.696529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a1216-ee71-416f-857e-97dc24c49a0a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:33.696857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a1216-ee71-416f-857e-97dc24c49a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:33.725158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c53a1216-ee71-416f-857e-97dc24c49a0a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:33.727277 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c53a1216-ee71-416f-857e-97dc24c49a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:33.772347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4649b2-4793-46f7-af7a-9e1242ef75a6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:33.808867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:33.809696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:33.809976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:33.810240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:33.821336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:33.823616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:33.830408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:33.830917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 363/715] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:33.832393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a1216-ee71-416f-857e-97dc24c49a0a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:33.845093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:33.846036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:33.846948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:33.846948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:33.846948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:33.846948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:33.846948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 353/716] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:33.851984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:33.918685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:33.965682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10153a6c-2393-46ab-b2de-cfb781c9617a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:33.972741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a1216-ee71-416f-857e-97dc24c49a0a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:34.005778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.006584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.006948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.007359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.008226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:34.008834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:34.011433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:34.024857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.025917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.026385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.026876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.027799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:34.028278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:34.029834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:34.031184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 354/717] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:34.032311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.038327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 364/718] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 17:12:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:34.039389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.122372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:34.166110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2278e8-c7cb-4e0d-89d0-78ac84b0077f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:34.172901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ecbb6-521d-4804-b731-d2490f6269dc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:34.203567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:34.204084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.205106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.205220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.205564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.217327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:34.218802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:34.243416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eef95-f321-4bd2-b24b-dfa9ae131b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.244677 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eef95-f321-4bd2-b24b-dfa9ae131b83 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.244959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eef95-f321-4bd2-b24b-dfa9ae131b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.245234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eef95-f321-4bd2-b24b-dfa9ae131b83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:34.248498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:34.249408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:34.250327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 365/719] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:34.260860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eef95-f321-4bd2-b24b-dfa9ae131b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:34.268344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ce5eef95-f321-4bd2-b24b-dfa9ae131b83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:34.270029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:34.270974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:34.271908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:34.275622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 355/720] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:34.284390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eef95-f321-4bd2-b24b-dfa9ae131b83 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.392224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664db50-11d3-4c2a-aa41-91f52c0a328d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:34.410754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eef95-f321-4bd2-b24b-dfa9ae131b83 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:34.419027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:34.434700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e56ffe-04b6-4d43-84a1-8224b85f21ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.435707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e56ffe-04b6-4d43-84a1-8224b85f21ce None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.436177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e56ffe-04b6-4d43-84a1-8224b85f21ce None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.436636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e56ffe-04b6-4d43-84a1-8224b85f21ce None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.437658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e56ffe-04b6-4d43-84a1-8224b85f21ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:34.438189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e56ffe-04b6-4d43-84a1-8224b85f21ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:34.439951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e56ffe-04b6-4d43-84a1-8224b85f21ce None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:34.453310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab6064-70f5-4bdd-b361-ac49998622a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.454065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6064-70f5-4bdd-b361-ac49998622a5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.454347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6064-70f5-4bdd-b361-ac49998622a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.454671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6064-70f5-4bdd-b361-ac49998622a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.455136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 366/721] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:34.455786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6064-70f5-4bdd-b361-ac49998622a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:34.456074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6064-70f5-4bdd-b361-ac49998622a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:34.457434 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6064-70f5-4bdd-b361-ac49998622a5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:34.457917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdf029-e11b-4a59-ac66-9ec61a8b1358 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.458416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdf029-e11b-4a59-ac66-9ec61a8b1358 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.458956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdf029-e11b-4a59-ac66-9ec61a8b1358 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.461289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 367/722] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:34.462238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdf029-e11b-4a59-ac66-9ec61a8b1358 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.469823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 356/723] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:34.471115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55aa4-9ade-40c5-b770-731c5e6bc6ac None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.474096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55aa4-9ade-40c5-b770-731c5e6bc6ac None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.474096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55aa4-9ade-40c5-b770-731c5e6bc6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.492033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fdc55aa4-9ade-40c5-b770-731c5e6bc6ac None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:34.493474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fdc55aa4-9ade-40c5-b770-731c5e6bc6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:34.534229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:34.579500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 357/724] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:34.580748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55aa4-9ade-40c5-b770-731c5e6bc6ac None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.600126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdf029-e11b-4a59-ac66-9ec61a8b1358 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:34.645440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.646482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.646760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.647031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.648237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=5b9888d2731f4117b5fc2c3a720931c4) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:34.648549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:34.651482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:34.667370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 368/725] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/role_assignments?user.id=5b9888d2731f4117b5fc2c3a720931c4&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=e07cc5009f624eb9912a837c934e68aa => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:34.667370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.719643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:34.785613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55aa4-9ade-40c5-b770-731c5e6bc6ac None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:34.793421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2355d-c6f3-4a34-b6a7-9b05220b2cfe None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:34.832189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7e99a-8f8f-42be-98b8-809e23783f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.833031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e99a-8f8f-42be-98b8-809e23783f23 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.833316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e99a-8f8f-42be-98b8-809e23783f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.833586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e99a-8f8f-42be-98b8-809e23783f23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.843374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:34.844210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:34.844485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:34.844757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:34.851189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7e99a-8f8f-42be-98b8-809e23783f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-1ed7e99a-8f8f-42be-98b8-809e23783f23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:34.859609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:34.860647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:34.861541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:34.862479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 369/726] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:34.870794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7e99a-8f8f-42be-98b8-809e23783f23 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.872286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:34.873765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:34.909421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:34.910412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:34.911316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:34.911316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:34.911316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:34.911316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:34.912090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 358/727] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:34.917298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:34.950283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:35.043253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4edfe-3a45-4f0a-8beb-2dab95208bdf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:35.082442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:35.083117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.083422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.083698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.084566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:35.084889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:35.086354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:35.087172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7e99a-8f8f-42be-98b8-809e23783f23 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:35.106742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:35.116390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 359/728] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:35.124332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:35.174714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:35.175404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.175687 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.175959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.176706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:35.177010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:35.178448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:35.187967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 370/729] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:35.196305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:35.253411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:35.282708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e937-62d6-44a7-b4f3-7e36cb13f74c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:35.336864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:35.338241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.338888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.339415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.354150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:35.358172 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:35.382773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:35.383832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:35.385147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:35.397041 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 360/730] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:12:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:35.405031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:35.412042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29d117-d213-49b9-b488-5405dd1f17ed None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:35.455283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:35.456462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.457180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.458091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.470610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:35.472488 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:35.492881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:35.494371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:35.495473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:35.495473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:35.495473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:35.495473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:35.502202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 371/731] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:35.503693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:35.529966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:35.617321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c65e5-eb36-46b9-a24a-36318c96b503 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:35.657055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98394ed4-cc5e-4117-8339-111d60987773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:35.657999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98394ed4-cc5e-4117-8339-111d60987773 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.658436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98394ed4-cc5e-4117-8339-111d60987773 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.658866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98394ed4-cc5e-4117-8339-111d60987773 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.659762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98394ed4-cc5e-4117-8339-111d60987773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:35.660215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98394ed4-cc5e-4117-8339-111d60987773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:35.662520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98394ed4-cc5e-4117-8339-111d60987773 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:35.682163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 372/732] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:35.683484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51314aeb-d971-43f8-92e1-9cca72fa30f3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.683770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51314aeb-d971-43f8-92e1-9cca72fa30f3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.684041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51314aeb-d971-43f8-92e1-9cca72fa30f3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.685477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 373/733] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:35.686012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51314aeb-d971-43f8-92e1-9cca72fa30f3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:35.704055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c33e-ffe6-4f58-9ade-8739e4e2ee33 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:35.751539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419de63-7a64-4ce7-a7c6-b089b3c11b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:35.752908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419de63-7a64-4ce7-a7c6-b089b3c11b26 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.753400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419de63-7a64-4ce7-a7c6-b089b3c11b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.754130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419de63-7a64-4ce7-a7c6-b089b3c11b26 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.755971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419de63-7a64-4ce7-a7c6-b089b3c11b26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:35.756510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419de63-7a64-4ce7-a7c6-b089b3c11b26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:35.759161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419de63-7a64-4ce7-a7c6-b089b3c11b26 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:35.783013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:35.787016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 361/734] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 17:12:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:35.789037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee735e-ef7f-42d0-89c5-05e3c7308494 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.789536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee735e-ef7f-42d0-89c5-05e3c7308494 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.790014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee735e-ef7f-42d0-89c5-05e3c7308494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.797219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:35.811204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51314aeb-d971-43f8-92e1-9cca72fa30f3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:35.814867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7aee735e-ef7f-42d0-89c5-05e3c7308494 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:35.816318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7aee735e-ef7f-42d0-89c5-05e3c7308494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:35.849180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:35.849976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:35.850252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:35.850767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:35.857909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:35.859625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:35.874848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:35.875850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:35.876791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:35.879619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 374/735] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:35.880446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:35.915987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 362/736] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:35.915987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee735e-ef7f-42d0-89c5-05e3c7308494 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:36.020719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51134e2-f059-4fea-ab02-0e9af23bc3e4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:36.042807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee735e-ef7f-42d0-89c5-05e3c7308494 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:36.067247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3caa-19d8-45f7-810b-e2fb4d897fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:36.068588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3caa-19d8-45f7-810b-e2fb4d897fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:36.069178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3caa-19d8-45f7-810b-e2fb4d897fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:36.069697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3caa-19d8-45f7-810b-e2fb4d897fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:36.076509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:36.081693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:36.082567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:36.083230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:36.084320 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:36.084827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:36.085527 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:36.087130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:36.088448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:36.117304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 363/737] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 17:12:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:36.117304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:36.176099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed3caa-19d8-45f7-810b-e2fb4d897fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:36.177627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed3caa-19d8-45f7-810b-e2fb4d897fa0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:36.240400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2470-48ed-44ab-95fa-dbbc41f8d365 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:36.285842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 375/738] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:36.285842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3caa-19d8-45f7-810b-e2fb4d897fa0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:36.296154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c66724-eb4a-49ad-a786-7688a15ab60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:36.297550 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c66724-eb4a-49ad-a786-7688a15ab60d None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:36.297982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c66724-eb4a-49ad-a786-7688a15ab60d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:36.298427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c66724-eb4a-49ad-a786-7688a15ab60d None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:36.299496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c66724-eb4a-49ad-a786-7688a15ab60d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:36.301674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c66724-eb4a-49ad-a786-7688a15ab60d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:36.324734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 364/739] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:12:36 2022] POST /identity/v3/services => generated 240 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:36.325964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c66724-eb4a-49ad-a786-7688a15ab60d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:36.371734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:36.474812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:36.500631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3caa-19d8-45f7-810b-e2fb4d897fa0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:36.549296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d2372-80b8-43ef-95f7-6742da8abe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:36.550758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d2372-80b8-43ef-95f7-6742da8abe78 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:36.551194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d2372-80b8-43ef-95f7-6742da8abe78 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:36.551656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d2372-80b8-43ef-95f7-6742da8abe78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:36.573577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c66724-eb4a-49ad-a786-7688a15ab60d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:36.620397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301101fa-15ef-4b17-88e7-6d34f0240940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:36.623395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301101fa-15ef-4b17-88e7-6d34f0240940 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:36.624111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301101fa-15ef-4b17-88e7-6d34f0240940 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:36.624804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301101fa-15ef-4b17-88e7-6d34f0240940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:36.643522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301101fa-15ef-4b17-88e7-6d34f0240940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:36.646603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301101fa-15ef-4b17-88e7-6d34f0240940 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:36.646785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d2372-80b8-43ef-95f7-6742da8abe78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:36.648457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d2372-80b8-43ef-95f7-6742da8abe78 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-301101fa-15ef-4b17-88e7-6d34f0240940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:36.679893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:36.682828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:36.684654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:36.684654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:36.684654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:36.684654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:36.686151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:36.688317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 365/740] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:36.697473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536778c-52ed-429f-b8bd-9d2047f8abec None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:36.698548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536778c-52ed-429f-b8bd-9d2047f8abec None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:36.699239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536778c-52ed-429f-b8bd-9d2047f8abec None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:36.702308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 366/741] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:36 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:36.703925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536778c-52ed-429f-b8bd-9d2047f8abec None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:36.839405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536778c-52ed-429f-b8bd-9d2047f8abec None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:36.841253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:36.881539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceba093-51cc-45dd-b053-d3ce5e54e64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:36.882373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceba093-51cc-45dd-b053-d3ce5e54e64b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:36.882757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceba093-51cc-45dd-b053-d3ce5e54e64b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:36.883153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceba093-51cc-45dd-b053-d3ce5e54e64b None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:36.884566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceba093-51cc-45dd-b053-d3ce5e54e64b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:36.885269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceba093-51cc-45dd-b053-d3ce5e54e64b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:36.887179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceba093-51cc-45dd-b053-d3ce5e54e64b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:36.907379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 367/742] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:36.907379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e233eb3-63ca-4446-99f1-badf6700fde7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:36.907379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e233eb3-63ca-4446-99f1-badf6700fde7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:36.907379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e233eb3-63ca-4446-99f1-badf6700fde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:36.921594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 376/743] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:36 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 17:12:36.923044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d2372-80b8-43ef-95f7-6742da8abe78 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:36.924498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5e233eb3-63ca-4446-99f1-badf6700fde7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:36.925791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5e233eb3-63ca-4446-99f1-badf6700fde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:37.010420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 368/744] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:36 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 17:12:37.011727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e233eb3-63ca-4446-99f1-badf6700fde7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.059685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d2372-80b8-43ef-95f7-6742da8abe78 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:37.102436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:37.104155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.104624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.105486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.109871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:37.116403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:37.118141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:37.123947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:37.135259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e233eb3-63ca-4446-99f1-badf6700fde7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:37.136439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:37.137348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:37.138246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:37.138246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:37.138246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:37.138246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:37.138246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:37.138246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:37.138246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:37.138246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:37.145662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 377/745] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:37.146771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.178748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:37.179572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.180096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.180096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.193309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:37.194706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:37.214347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:37.215266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:37.216865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:37.216865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:37.216865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:37.216865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:37.216865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 369/746] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:37.216865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.257429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd91a13-4749-4017-b034-1f01d9a9f5ef None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:37.292168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eae718-a5a1-4a91-a379-5cced60f771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:37.292883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae718-a5a1-4a91-a379-5cced60f771d None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.293168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae718-a5a1-4a91-a379-5cced60f771d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.293440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae718-a5a1-4a91-a379-5cced60f771d None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.294159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae718-a5a1-4a91-a379-5cced60f771d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:37.295559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae718-a5a1-4a91-a379-5cced60f771d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:37.310918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 378/747] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:12:37 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:37.312319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1aefdc-e4af-4ad7-b74f-23e2c4bda672 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.312595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1aefdc-e4af-4ad7-b74f-23e2c4bda672 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.312874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1aefdc-e4af-4ad7-b74f-23e2c4bda672 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.318684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 379/748] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:37.319667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1aefdc-e4af-4ad7-b74f-23e2c4bda672 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.347561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056bca9-4a8b-47a1-a193-20c53f3d0c5f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:37.392435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:37.423503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fec3e-fa32-4aad-85f6-47d2099bab85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:37.425000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fec3e-fa32-4aad-85f6-47d2099bab85 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.425237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fec3e-fa32-4aad-85f6-47d2099bab85 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.425526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fec3e-fa32-4aad-85f6-47d2099bab85 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.428100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fec3e-fa32-4aad-85f6-47d2099bab85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:37.428709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fec3e-fa32-4aad-85f6-47d2099bab85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:37.431289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fec3e-fa32-4aad-85f6-47d2099bab85 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:37.444239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:37.455660 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 370/749] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:37.457572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a050e2-70a4-4785-adc5-0e560876d9e8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.457927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a050e2-70a4-4785-adc5-0e560876d9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.458354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a050e2-70a4-4785-adc5-0e560876d9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.459798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1aefdc-e4af-4ad7-b74f-23e2c4bda672 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:37.483716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-30a050e2-70a4-4785-adc5-0e560876d9e8 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:37.485310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-30a050e2-70a4-4785-adc5-0e560876d9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:37.503301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:37.504009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.504293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.504571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.505302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:37.505613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:37.507025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:37.519751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 380/750] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:37.520955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.586608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 371/751] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:37.588324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a050e2-70a4-4785-adc5-0e560876d9e8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.651039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9574-ceb3-4331-8da2-0116ee9de947 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:37.668204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:37.687300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:37.687779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.688060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.688333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.689167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=ba50db32ce9f444e919e667305594d4a) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:37.689480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:37.690929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:37.696968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 381/752] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:37 2022] GET /identity/v3/role_assignments?user.id=ba50db32ce9f444e919e667305594d4a&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:37.697940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.723376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a050e2-70a4-4785-adc5-0e560876d9e8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:37.759249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:37.777410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:37.778672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.779314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.779848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.791648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:37.799910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:37.826274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:37.827258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:37.828220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:37.828220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:37.828220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:37.828220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:37.828220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:37.828220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:37.828220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:37.828220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:37.834768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 372/753] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:37.840452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.841822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e56e63-ab21-4abb-86a5-61a19dd4a00f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:37.879409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ea7e3-960b-4a82-b751-8ca0542e79db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:37.880225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ea7e3-960b-4a82-b751-8ca0542e79db None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:37.880505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ea7e3-960b-4a82-b751-8ca0542e79db None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:37.880779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ea7e3-960b-4a82-b751-8ca0542e79db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:37.899116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ea7e3-960b-4a82-b751-8ca0542e79db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e50ea7e3-960b-4a82-b751-8ca0542e79db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:37.899750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:37.901365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:37.902692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 382/754] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:37.908424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ea7e3-960b-4a82-b751-8ca0542e79db None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:37.963718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:38.024189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ea7e3-960b-4a82-b751-8ca0542e79db None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:38.064321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:38.065146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.065430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.065702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.066592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:38.066902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:38.068772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:38.082704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 383/755] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:38.084686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55830f-4f6b-4194-a3d7-c46b14ac659b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:38.094442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:38.153585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:38.154186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.154471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.154749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.155583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:38.155896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:38.157339 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:38.170406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 373/756] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:38.180603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:38.237059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018daa8a-9b7f-45a8-b12a-6037a8931660 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:38.240240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:38.291223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:38.292066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.292630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:38.293063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.293346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.305769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:38.307452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:38.325974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:38.326894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:38.327820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:38.327820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:38.327820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:38.327820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:38.331648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 384/757] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:38.333035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:38.397015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93177a0f-5d0f-4583-b1a0-83daf1b94a96 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:38.442700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06888f68-9531-41e7-a6d5-15f464269be1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:38.458151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:38.458903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.459184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.459488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.460595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=c4224f58ecbb423dae16d5b9ae09d5cb) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:38.460733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:38.462119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:38.475645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 374/758] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:38 2022] GET /identity/v3/role_assignments?user.id=c4224f58ecbb423dae16d5b9ae09d5cb&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=e07cc5009f624eb9912a837c934e68aa => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:38.480040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:38.480937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.481371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.481791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.482698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:38.483151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:38.485002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:38.500630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 385/759] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:38.506605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:38.661618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75c800-3bca-4356-8b16-a25ca7701e31 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:38.695748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:38.707056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1093311-2867-475b-af43-e146f36fe787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:38.707961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1093311-2867-475b-af43-e146f36fe787 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.708249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1093311-2867-475b-af43-e146f36fe787 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.708525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1093311-2867-475b-af43-e146f36fe787 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.718508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1093311-2867-475b-af43-e146f36fe787 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:38.720021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1093311-2867-475b-af43-e146f36fe787 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c1093311-2867-475b-af43-e146f36fe787 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:38.743289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:38.744425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:38.745387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:38.749028 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 375/760] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:12:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:38.754792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:38.779759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:38.827274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d9927-bb14-437d-a27d-52441113e672 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.827634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d9927-bb14-437d-a27d-52441113e672 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.827862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d9927-bb14-437d-a27d-52441113e672 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.829431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 376/761] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:12:38.834966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f08bf-75e6-4fd3-b176-6acdba6c8fbf None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.835642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f08bf-75e6-4fd3-b176-6acdba6c8fbf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.836151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f08bf-75e6-4fd3-b176-6acdba6c8fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.854555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0f2f08bf-75e6-4fd3-b176-6acdba6c8fbf None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:38.856574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0f2f08bf-75e6-4fd3-b176-6acdba6c8fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:38.865882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71663c36-ddcf-4323-9c4a-9545dd7620ae None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:38.920933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:38.921608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:38.921888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:38.922158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:38.922999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:38.923302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:38.924751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:38.963931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 386/762] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:12:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:38.970972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:38.988242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 377/763] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:39.026253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f08bf-75e6-4fd3-b176-6acdba6c8fbf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:39.067708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:39.114378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348d442-ef15-432e-aed8-139bade3eea7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:39.188790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:39.189592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:39.189869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:39.190136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:39.211194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:39.213495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:39.222483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f08bf-75e6-4fd3-b176-6acdba6c8fbf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:39.229753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:39.253800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:39.254772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:39.255718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:39.255718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:39.255718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:39.255718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:39.257552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 387/764] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:39.261609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:39.289571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:39.291647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:39.291931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:39.292210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:39.292873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:39.294240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:39.331911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:39.332964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:39.334641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 378/765] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:39 2022] GET /identity/v3/services/network => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:39.344242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:39.384130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84dec4-b94d-43b1-ba26-2bc92096fedc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:39.412920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:39.421148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:39.421825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:39.422154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:39.422564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:39.423380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:39.423838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:39.425576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:39.438624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 388/766] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:39.445030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:39.549403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16759d9-a13d-4b1e-8c9d-8fc23a71d999 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:39.564621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5423d-8752-40d2-8c6f-1501e9259bd3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:39.586261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:39.587122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:39.587526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:39.587804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:39.595581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:39.597040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:39.613628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:39.614594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:39.615988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 389/767] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:39.624401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:39.653413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:39.654626 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:39.655996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:39.657223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:39.658170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] RBAC: Adding query filter params (name=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:39.658907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:39.661900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:39.676956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 379/768] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:12:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:39.681949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:39.725091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:39.737993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:39.854182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ee157-e1f0-4e17-8bd1-28be59c6560b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:39.858619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdd72b-5d6e-41ba-b3a6-71e73e3e4df1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:39.907091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:39.908789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:39.909064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:39.909338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:39.910181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:39.910474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:39.911969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:39.936713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:39.937393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:39.937678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:39.937954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:39.938669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] RBAC: Adding query filter params (type=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:39.939708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:39.941101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:39.942492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 390/769] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:39.948505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:39.955362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 380/770] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:12:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:39.958954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:40.096182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:40.097172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9559a02-11ec-44ca-a623-c08ac328dfaf None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:40.099629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:40.105720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a77a3-5495-4142-a355-2d2cbd96e795 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:40.136512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a4f5-4499-4cfc-a970-c521318f1021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:40.137506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4f5-4499-4cfc-a970-c521318f1021 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.137792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4f5-4499-4cfc-a970-c521318f1021 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.138068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4f5-4499-4cfc-a970-c521318f1021 None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/5b9888d2731f4117b5fc2c3a720931c4/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.140455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a4f5-4499-4cfc-a970-c521318f1021 None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=5b9888d2731f4117b5fc2c3a720931c4, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:40.142053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a4f5-4499-4cfc-a970-c521318f1021 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:40.143327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd22396-6164-45c9-9fce-2473c60aebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:40.144097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22396-6164-45c9-9fce-2473c60aebeb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.144556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22396-6164-45c9-9fce-2473c60aebeb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.144896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22396-6164-45c9-9fce-2473c60aebeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.145875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22396-6164-45c9-9fce-2473c60aebeb None admin] RBAC: Adding query filter params (interface=public, service_id=47eec42e018c4d908cfa14c1a759216f, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:40.146255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22396-6164-45c9-9fce-2473c60aebeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:40.147946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22396-6164-45c9-9fce-2473c60aebeb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:40.158289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 381/771] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 17:12:39 2022] GET /identity/v3/endpoints?service_id=47eec42e018c4d908cfa14c1a759216f&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:40.160084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e95cd-b958-4747-88a8-5217706dd6fc None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.160239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e95cd-b958-4747-88a8-5217706dd6fc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.160526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e95cd-b958-4747-88a8-5217706dd6fc None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.169846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 382/772] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:40.170053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 391/773] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:39 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/5b9888d2731f4117b5fc2c3a720931c4/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:12:40.171734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e95cd-b958-4747-88a8-5217706dd6fc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:40.178322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c782e-4470-4dbc-8f6c-3176e1b2d38a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.178984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c782e-4470-4dbc-8f6c-3176e1b2d38a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.179519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c782e-4470-4dbc-8f6c-3176e1b2d38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.222621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-874c782e-4470-4dbc-8f6c-3176e1b2d38a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:40.225386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-874c782e-4470-4dbc-8f6c-3176e1b2d38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:40.375967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 392/774] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:40.376157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e95cd-b958-4747-88a8-5217706dd6fc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:40.376463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f63f3-06de-45e3-bbf7-c9ac825e1095 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.377003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f63f3-06de-45e3-bbf7-c9ac825e1095 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.377281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f63f3-06de-45e3-bbf7-c9ac825e1095 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.383509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 393/775] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:40.386216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe637bab-06b9-4e42-816b-20e023c00cf3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.386503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe637bab-06b9-4e42-816b-20e023c00cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.387245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe637bab-06b9-4e42-816b-20e023c00cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.411634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-fe637bab-06b9-4e42-816b-20e023c00cf3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:40.413110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fe637bab-06b9-4e42-816b-20e023c00cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:40.424990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2c9bd-d5c0-44a0-ab8a-f203ea44cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:40.425984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c9bd-d5c0-44a0-ab8a-f203ea44cb97 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.426509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c9bd-d5c0-44a0-ab8a-f203ea44cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.426708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c9bd-d5c0-44a0-ab8a-f203ea44cb97 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.427474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2c9bd-d5c0-44a0-ab8a-f203ea44cb97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:40.428978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2c9bd-d5c0-44a0-ab8a-f203ea44cb97 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:40.435199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 383/776] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:12:40 2022] GET /identity/v3/services => generated 772 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:40.436024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2c9bd-d5c0-44a0-ab8a-f203ea44cb97 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:40.471800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:40.504591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 394/777] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:40.528620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe637bab-06b9-4e42-816b-20e023c00cf3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:40.581767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:40.625472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2c9bd-d5c0-44a0-ab8a-f203ea44cb97 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:40.671089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:40.671872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.672150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.672421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.674002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:40.679068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:40.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:40.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:40.698365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:40.702755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 384/778] 10.222.0.33 () {56 vars in 1129 bytes} [Sat May 28 17:12:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:40.704075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:40.731088 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe637bab-06b9-4e42-816b-20e023c00cf3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:40.769519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2368910-a613-4afa-b3e8-8ce0447be17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:40.770721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368910-a613-4afa-b3e8-8ce0447be17d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.771244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368910-a613-4afa-b3e8-8ce0447be17d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.771741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368910-a613-4afa-b3e8-8ce0447be17d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.784663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2368910-a613-4afa-b3e8-8ce0447be17d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b2368910-a613-4afa-b3e8-8ce0447be17d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:40.792978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:40.794046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:40.794891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:40.797907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 395/779] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:40.803767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2368910-a613-4afa-b3e8-8ce0447be17d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:40.814336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:40.831205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5008e6-bf8e-468f-b939-1bad0d3e67ae None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:40.885815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:40.886851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:40.887357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:40.887862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:40.888779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:40.889432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:40.892736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:40.902489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 385/780] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 17:12:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:40.911267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:40.955132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:40.984938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2368910-a613-4afa-b3e8-8ce0447be17d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.045829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e6378-f069-473f-909c-d80bdee8f6c3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.073814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.074495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.074775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.075046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.075823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:41.076127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:41.077505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:41.081865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.082894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.083437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.083904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.084782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:41.085271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:41.086840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:41.093060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 386/781] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 17:12:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:41.097774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 396/782] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:41.099231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:41.100610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:41.123770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:41.226478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728770f1-7828-47d4-a74d-6e59a3f988a9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.235921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b92705-bf2d-4eb0-866d-3777531535b8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.269648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559cca9-3286-4b4a-871a-0b083eabb471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.271114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559cca9-3286-4b4a-871a-0b083eabb471 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.271431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559cca9-3286-4b4a-871a-0b083eabb471 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.271709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559cca9-3286-4b4a-871a-0b083eabb471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.278207 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06248bce-41d3-44da-90a1-e5bafd2f4ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.278941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248bce-41d3-44da-90a1-e5bafd2f4ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.279287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248bce-41d3-44da-90a1-e5bafd2f4ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.279661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248bce-41d3-44da-90a1-e5bafd2f4ae7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.280539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06248bce-41d3-44da-90a1-e5bafd2f4ae7 None admin] RBAC: Adding query filter params (interface=public, service_id=021e7a90c72c4b11ba476e7a4af571e4, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:41.280915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06248bce-41d3-44da-90a1-e5bafd2f4ae7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:41.282356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06248bce-41d3-44da-90a1-e5bafd2f4ae7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:41.286698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559cca9-3286-4b4a-871a-0b083eabb471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:41.288166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559cca9-3286-4b4a-871a-0b083eabb471 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:41.288279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 387/783] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/endpoints?service_id=021e7a90c72c4b11ba476e7a4af571e4&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:41.289549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d197ded-9a59-47db-ab0f-eea4342d200b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.289901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d197ded-9a59-47db-ab0f-eea4342d200b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.290243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d197ded-9a59-47db-ab0f-eea4342d200b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.292780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 388/784] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:41.294563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d197ded-9a59-47db-ab0f-eea4342d200b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:41.299087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c559cca9-3286-4b4a-871a-0b083eabb471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:41.318458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:41.319622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:41.320732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:41.320732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:41.320732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:41.320732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:41.323522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 397/785] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:41.325571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2464a1-d5c3-4905-a904-f98852ea6ccc None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.326008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2464a1-d5c3-4905-a904-f98852ea6ccc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.326580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2464a1-d5c3-4905-a904-f98852ea6ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.346059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1f2464a1-d5c3-4905-a904-f98852ea6ccc None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:41.347388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1f2464a1-d5c3-4905-a904-f98852ea6ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:41.405465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:41.427833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 398/786] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:41.429197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2464a1-d5c3-4905-a904-f98852ea6ccc None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:41.429689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d197ded-9a59-47db-ab0f-eea4342d200b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.470814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7424ba-8dee-41e0-9a1c-9a89167c5d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.471778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7424ba-8dee-41e0-9a1c-9a89167c5d25 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.472154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7424ba-8dee-41e0-9a1c-9a89167c5d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.472806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7424ba-8dee-41e0-9a1c-9a89167c5d25 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.473312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7424ba-8dee-41e0-9a1c-9a89167c5d25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:41.477794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7424ba-8dee-41e0-9a1c-9a89167c5d25 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:41.482031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 389/787] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:41.483176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7424ba-8dee-41e0-9a1c-9a89167c5d25 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:41.567649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2464a1-d5c3-4905-a904-f98852ea6ccc None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.597503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7424ba-8dee-41e0-9a1c-9a89167c5d25 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.599128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:41.610567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.611549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.611980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.612401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.613438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:41.613900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:41.615628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:41.629593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 399/788] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:41.635034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:41.636995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120936-ca5a-4ec8-97ca-f74303c027a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.637885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120936-ca5a-4ec8-97ca-f74303c027a9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.638234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120936-ca5a-4ec8-97ca-f74303c027a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.638580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120936-ca5a-4ec8-97ca-f74303c027a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.652390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b120936-ca5a-4ec8-97ca-f74303c027a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8b120936-ca5a-4ec8-97ca-f74303c027a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:41.660891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:41.661889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:41.668916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 390/789] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:41.672505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120936-ca5a-4ec8-97ca-f74303c027a9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:41.694018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:41.826828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120936-ca5a-4ec8-97ca-f74303c027a9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.835142 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b64fff-df48-4a1f-b04d-581fc2ece795 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:41.868039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.869950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.870387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.870997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.872440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:41.873152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:41.875867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:41.878808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:41.880205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:41.880680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:41.881159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:41.887376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 391/790] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:41.891468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:41.893442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:41.898888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:41.911196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:41.912221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:41.913623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:41.914376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 400/791] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:41.921834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:42.037113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:42.046733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:42.095557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ac741-0fa1-496f-aa19-47c721271a8b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:42.139366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:42.139366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.139366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.139366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.147803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:42.149393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:42.169507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:42.170465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:42.171831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:42.171831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:42.171831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:42.171831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:42.171831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 392/792] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:42.176931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:42.177753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af7aa2-f9af-4798-bd94-9da5872eec31 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:42.258261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:42.258944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.259517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.259947 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.261309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:42.261621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:42.263073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:42.292698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e646a75-fec3-48f3-bef6-3ee2701ee16f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:42.302172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 401/793] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 17:12:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:42.308527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:42.319783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:42.339465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:42.340398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.340825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.341367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.342249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:42.342802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:42.344514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:42.374345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 393/794] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:42.376108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:42.521992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f60740-d541-4c53-ae34-f8672ae2fcf6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:42.539814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461b70-9f99-4280-b2a0-a58e0993f490 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:42.551764 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:42.605618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:42.606722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.607242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.607673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.619790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:42.620979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.621120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:42.621525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.622003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.622718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:42.635445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:42.637305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:42.653440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:42.654746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:42.655966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:42.661115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 394/795] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:42.676086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:42.677226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:42.678341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:42.678341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:42.678341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:42.678341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:42.680965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:42.692856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 402/796] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:42.694455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:42.763916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:42.843423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54fac7-c89d-46d9-a8c1-630f21d28896 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:42.865954 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90441f21-a878-45e6-b0ec-8558ba487d1a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:42.881163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b6ee0-cebf-4ee7-ae27-7dec5e45c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:42.881843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b6ee0-cebf-4ee7-ae27-7dec5e45c06b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.882121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b6ee0-cebf-4ee7-ae27-7dec5e45c06b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.882497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b6ee0-cebf-4ee7-ae27-7dec5e45c06b None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.883367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b6ee0-cebf-4ee7-ae27-7dec5e45c06b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:42.883678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b6ee0-cebf-4ee7-ae27-7dec5e45c06b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:42.885609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b6ee0-cebf-4ee7-ae27-7dec5e45c06b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:42.906756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 395/797] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 17:12:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:42.912697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f643340-d536-413b-9fd3-d22e4e0024db None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.912987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f643340-d536-413b-9fd3-d22e4e0024db None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.913267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f643340-d536-413b-9fd3-d22e4e0024db None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.914872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 396/798] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:42.916798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f643340-d536-413b-9fd3-d22e4e0024db None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:42.923420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ade22-ebb9-4e00-8f0a-4be28e7e2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:42.924113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ade22-ebb9-4e00-8f0a-4be28e7e2b76 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.924393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ade22-ebb9-4e00-8f0a-4be28e7e2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.924667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ade22-ebb9-4e00-8f0a-4be28e7e2b76 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.925402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ade22-ebb9-4e00-8f0a-4be28e7e2b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:42.925709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ade22-ebb9-4e00-8f0a-4be28e7e2b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:42.927636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ade22-ebb9-4e00-8f0a-4be28e7e2b76 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:42.947265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 403/799] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:42.948665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbad85-a330-4aa3-8293-30b8ca00d325 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:42.948951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbad85-a330-4aa3-8293-30b8ca00d325 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:42.949226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbad85-a330-4aa3-8293-30b8ca00d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:42.970979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9afbad85-a330-4aa3-8293-30b8ca00d325 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:42.972172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9afbad85-a330-4aa3-8293-30b8ca00d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:42.987626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:43.051362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 404/800] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:43.051362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbad85-a330-4aa3-8293-30b8ca00d325 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.068727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:43.081201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f643340-d536-413b-9fd3-d22e4e0024db None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.134008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.135545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.136117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.136835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.154103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:43.156803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:43.174251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbad85-a330-4aa3-8293-30b8ca00d325 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:43.182749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:43.183768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:43.184680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:43.184680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:43.184680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:43.184680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:43.185871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 397/801] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:43.185871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.225760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.227449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.229405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.229841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.238650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:43.240675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:43.259940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:43.260989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:43.262114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:43.262114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:43.262114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:43.262114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:43.262114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:43.262114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:43.262114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:43.262114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:43.268920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 405/802] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:43.268920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.283675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:43.309211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a19d2-e529-49cc-986e-ab781faa87ff None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.347218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bc966-1ae0-4c99-bc6e-940663c945ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.348445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bc966-1ae0-4c99-bc6e-940663c945ab None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.348885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bc966-1ae0-4c99-bc6e-940663c945ab None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.349303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bc966-1ae0-4c99-bc6e-940663c945ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.360371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bc966-1ae0-4c99-bc6e-940663c945ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-df6bc966-1ae0-4c99-bc6e-940663c945ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:43.367973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:43.368953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:43.369809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 398/803] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:43.370501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bc966-1ae0-4c99-bc6e-940663c945ab None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.385277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:43.395425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a24e3-afb0-440a-ac61-4462a5f49b86 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.435644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833488b-091d-44da-9455-a30753e07d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.436202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833488b-091d-44da-9455-a30753e07d8b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.436481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833488b-091d-44da-9455-a30753e07d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.436749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833488b-091d-44da-9455-a30753e07d8b None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.437541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833488b-091d-44da-9455-a30753e07d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:43.437815 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833488b-091d-44da-9455-a30753e07d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:43.439127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833488b-091d-44da-9455-a30753e07d8b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:43.452143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 406/804] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:43.453383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3493d-c335-47c2-bf14-9a8fbcb1b120 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.453669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3493d-c335-47c2-bf14-9a8fbcb1b120 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.453943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3493d-c335-47c2-bf14-9a8fbcb1b120 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.459694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 407/805] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:43.460503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3493d-c335-47c2-bf14-9a8fbcb1b120 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.513850 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bc966-1ae0-4c99-bc6e-940663c945ab None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.544430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:43.550291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.550956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.551238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.551539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.552330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:43.552639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:43.554494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:43.573638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 399/806] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:43.573638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.585302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3493d-c335-47c2-bf14-9a8fbcb1b120 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.621159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb5542-8809-414b-a2ab-85b08b6938db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.621828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb5542-8809-414b-a2ab-85b08b6938db None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.622110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb5542-8809-414b-a2ab-85b08b6938db None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.622494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb5542-8809-414b-a2ab-85b08b6938db None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.623267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb5542-8809-414b-a2ab-85b08b6938db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:43.623613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb5542-8809-414b-a2ab-85b08b6938db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:43.625067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb5542-8809-414b-a2ab-85b08b6938db None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:43.638237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 408/807] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:43.638237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e11cba-830b-40e8-a6f8-1204c6bdc466 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.638237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e11cba-830b-40e8-a6f8-1204c6bdc466 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.638237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e11cba-830b-40e8-a6f8-1204c6bdc466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.651248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:43.654876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-78e11cba-830b-40e8-a6f8-1204c6bdc466 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:43.656102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-78e11cba-830b-40e8-a6f8-1204c6bdc466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:43.687571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd304cf-8f2d-4f3d-9377-0b54f25fc4fa None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.739364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 409/808] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:43.739555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.739666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e11cba-830b-40e8-a6f8-1204c6bdc466 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.739752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.739752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.739752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.749479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:43.750929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:43.766949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:43.767925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:43.768896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 400/809] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:43.769387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.795328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:43.840259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e11cba-830b-40e8-a6f8-1204c6bdc466 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.879148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e80ab5-8343-44d0-a96e-0e9cd57bf90b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:43.883211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b67b6-8399-4420-8bcb-fdc14078bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.884173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b67b6-8399-4420-8bcb-fdc14078bb13 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.884456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b67b6-8399-4420-8bcb-fdc14078bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.884729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b67b6-8399-4420-8bcb-fdc14078bb13 None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/ba50db32ce9f444e919e667305594d4a/roles/7dd141eb611c4f98a92d16b8b21b7a9d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.901188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b67b6-8399-4420-8bcb-fdc14078bb13 None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=ba50db32ce9f444e919e667305594d4a, role_id=7dd141eb611c4f98a92d16b8b21b7a9d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:43.902763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b67b6-8399-4420-8bcb-fdc14078bb13 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:43.908075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:43.917736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:43.927577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 410/810] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:43 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/ba50db32ce9f444e919e667305594d4a/roles/7dd141eb611c4f98a92d16b8b21b7a9d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:12:43.927788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:43.927788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:43.927957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b67b6-8399-4420-8bcb-fdc14078bb13 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:43.928059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:43.940167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:43.941752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:43.970242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:43.971241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:43.972170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:43.972170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:43.972170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:43.972170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:43.973521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 401/811] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:43.974149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.062856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b67b6-8399-4420-8bcb-fdc14078bb13 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.079460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:44.108921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04471426-1fc2-496e-bd5f-849f176505bf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.117261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.118804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.119086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.119384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] PATH_INFO: `/v3/services/network` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.120060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.121949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:44.143570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:44.144653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:44.145492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 411/812] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:44.146302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.162070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.162676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.162961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.163238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.164805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:44.165120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.167189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.186845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 402/813] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:44.186845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.242428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:44.275853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14a46-4558-4912-b7dd-ec32ce37626e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.310138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.310808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.311101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.311391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.312121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:44.312429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.313831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.317126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cee7b7-5aa5-4112-929e-59d3252052e3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.326567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 412/814] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:44.327383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.360213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5e3ef-6ff7-4764-8e89-f4c873fb4890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.360828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5e3ef-6ff7-4764-8e89-f4c873fb4890 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.361157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5e3ef-6ff7-4764-8e89-f4c873fb4890 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.361485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5e3ef-6ff7-4764-8e89-f4c873fb4890 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.362201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5e3ef-6ff7-4764-8e89-f4c873fb4890 None admin] RBAC: Adding query filter params (name=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:44.362761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5e3ef-6ff7-4764-8e89-f4c873fb4890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.363897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5e3ef-6ff7-4764-8e89-f4c873fb4890 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.369741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 403/815] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:12:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:44.370064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:44.370718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f3f66-ab3f-47ab-bbc4-f5fd63ea88c1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.371060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f3f66-ab3f-47ab-bbc4-f5fd63ea88c1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.371295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f3f66-ab3f-47ab-bbc4-f5fd63ea88c1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.377121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 404/816] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:44.377121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f3f66-ab3f-47ab-bbc4-f5fd63ea88c1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.446199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70364ef5-38ca-4e25-9970-33662678136a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.483850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61c80e-f2f7-4317-8457-ea6e2321a227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.484996 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61c80e-f2f7-4317-8457-ea6e2321a227 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.485275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61c80e-f2f7-4317-8457-ea6e2321a227 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.485587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61c80e-f2f7-4317-8457-ea6e2321a227 None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/c4224f58ecbb423dae16d5b9ae09d5cb/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.487568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61c80e-f2f7-4317-8457-ea6e2321a227 None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=c4224f58ecbb423dae16d5b9ae09d5cb, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.488971 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61c80e-f2f7-4317-8457-ea6e2321a227 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.494470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f3f66-ab3f-47ab-bbc4-f5fd63ea88c1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.497451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:44.507412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 413/817] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:44 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/c4224f58ecbb423dae16d5b9ae09d5cb/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:12:44.508291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61c80e-f2f7-4317-8457-ea6e2321a227 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.547620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef9cee-a109-4e7e-9ad8-5a2c4390cc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.548427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef9cee-a109-4e7e-9ad8-5a2c4390cc27 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.548772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef9cee-a109-4e7e-9ad8-5a2c4390cc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.549057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef9cee-a109-4e7e-9ad8-5a2c4390cc27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.558111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef9cee-a109-4e7e-9ad8-5a2c4390cc27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.559707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef9cee-a109-4e7e-9ad8-5a2c4390cc27 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a8ef9cee-a109-4e7e-9ad8-5a2c4390cc27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:44.576474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:44.577530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:12:44.578516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:44.583012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 405/818] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:12:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:44.584576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bcee9-a704-4e8b-9d60-2e37138a51d4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.584868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bcee9-a704-4e8b-9d60-2e37138a51d4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.585146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bcee9-a704-4e8b-9d60-2e37138a51d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.607600 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-547bcee9-a704-4e8b-9d60-2e37138a51d4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:44.609869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-547bcee9-a704-4e8b-9d60-2e37138a51d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:44.638304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:44.647655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61c80e-f2f7-4317-8457-ea6e2321a227 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.703440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.703829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 406/819] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:44.704044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.704331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.704604 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.704704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bcee9-a704-4e8b-9d60-2e37138a51d4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.705325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] RBAC: Adding query filter params (type=network) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:44.705633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.707365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.719430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 414/820] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:12:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:44.728148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.829486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:44.840594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bcee9-a704-4e8b-9d60-2e37138a51d4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.862070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2119f-2bc5-4a96-8ed9-002ef6ad9110 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:44.887504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.888927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.889307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.889943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.890906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:44.891587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.893830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.904700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787df53a-cf59-4537-adf5-7e6964fecb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:44.905686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787df53a-cf59-4537-adf5-7e6964fecb6f None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:44.906107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787df53a-cf59-4537-adf5-7e6964fecb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:44.906630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787df53a-cf59-4537-adf5-7e6964fecb6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:44.907618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787df53a-cf59-4537-adf5-7e6964fecb6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:44.909179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787df53a-cf59-4537-adf5-7e6964fecb6f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:44.917690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 407/821] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:12:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:44.918672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.934732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 415/822] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:12:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:44.936193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787df53a-cf59-4537-adf5-7e6964fecb6f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:44.958196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:45.061720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787df53a-cf59-4537-adf5-7e6964fecb6f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:45.091765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ae847-90a7-4515-a7c3-70c110af5243 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:45.102418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a81c6-b19e-4918-a432-a8c02b3155c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:45.104451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a81c6-b19e-4918-a432-a8c02b3155c5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:45.104967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a81c6-b19e-4918-a432-a8c02b3155c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:45.105454 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a81c6-b19e-4918-a432-a8c02b3155c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:45.179732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:45.180573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:45.180857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:45.181403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:45.182084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:45.184538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:45.206420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:45.207494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 17:12:45.208384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:45.209726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 408/823] 10.222.0.33 () {56 vars in 1129 bytes} [Sat May 28 17:12:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:45.219786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:45.230553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a81c6-b19e-4918-a432-a8c02b3155c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:45.232211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a81c6-b19e-4918-a432-a8c02b3155c5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:45.252204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:45.259693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:45.380381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562594c2-1243-4159-8f9c-5edaedf870f1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:45.393208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 416/824] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:44 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:12:45.400280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a81c6-b19e-4918-a432-a8c02b3155c5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:45.422012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:45.422696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:45.423277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:45.423588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:45.424308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:45.424621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:45.426455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:45.433322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 409/825] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 17:12:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:45.438930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:45.525730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a81c6-b19e-4918-a432-a8c02b3155c5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:45.536719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:45.548703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:45.565972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:45.567695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:45.568132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:45.568808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:45.572802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1f922-36d9-405b-9c43-57de1602f8d8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:45.582037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:45.584178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:45.607422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:45.608521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:45.609776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:45.609776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:45.609776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:45.609776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:45.609776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 417/826] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:45.615524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:45.625814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:45.626578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:45.626932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:45.627287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:45.628459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:45.628861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:45.630629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:45.637450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 410/827] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 17:12:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:45.642465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:45.744390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952634e-4edb-43c4-8b8c-1dbd9850dbf8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:45.768165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1a5bf-5824-45b0-bdce-8109e8e23647 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:45.787799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:45.788409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:45.788693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:45.788976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:45.789735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:45.790257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:45.792912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:45.812035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ce49-850c-4ec9-8b34-a30a83d100e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:45.813225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 418/828] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:45.813692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ce49-850c-4ec9-8b34-a30a83d100e9 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:45.814121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:45.814398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ce49-850c-4ec9-8b34-a30a83d100e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:45.814864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ce49-850c-4ec9-8b34-a30a83d100e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:45.815946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860ce49-850c-4ec9-8b34-a30a83d100e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:45.818654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860ce49-850c-4ec9-8b34-a30a83d100e9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:45.838930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:45.844894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:45.855571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 411/829] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:12:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:12:45.945375 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ea059-3ad9-427c-9283-2aaf11c6805e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:45.990848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:45.991778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:45.992067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:45.992342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.000254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:46.001771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:46.017424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:46.018460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:46.019785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 419/830] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:46.025543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ce49-850c-4ec9-8b34-a30a83d100e9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:46.142337 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ce49-850c-4ec9-8b34-a30a83d100e9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:46.177131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276cb41-c0de-4cce-b9e3-db11627dff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:46.177795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276cb41-c0de-4cce-b9e3-db11627dff3a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.178078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276cb41-c0de-4cce-b9e3-db11627dff3a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.178357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276cb41-c0de-4cce-b9e3-db11627dff3a None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.179142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276cb41-c0de-4cce-b9e3-db11627dff3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:46.179474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276cb41-c0de-4cce-b9e3-db11627dff3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:46.180913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276cb41-c0de-4cce-b9e3-db11627dff3a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:46.195155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 412/831] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:46.202277 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:46.286498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:46.293261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7f1d-16a7-49eb-aa5f-bf2fa6692091 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.293552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7f1d-16a7-49eb-aa5f-bf2fa6692091 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.293834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7f1d-16a7-49eb-aa5f-bf2fa6692091 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.295360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 413/832] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:46.300413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33152a-5031-49e0-b2ca-bbdf990bceed None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.300699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33152a-5031-49e0-b2ca-bbdf990bceed None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.300977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33152a-5031-49e0-b2ca-bbdf990bceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.312382 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ee227-f5fb-4f71-91a5-cccf925edaf2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:46.325470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bd33152a-5031-49e0-b2ca-bbdf990bceed None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:46.326956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bd33152a-5031-49e0-b2ca-bbdf990bceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:46.361482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:46.362153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.362438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.362715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.363598 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=5b9888d2731f4117b5fc2c3a720931c4) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:46.363904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:46.365358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:46.383386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 420/833] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:46 2022] GET /identity/v3/role_assignments?user.id=5b9888d2731f4117b5fc2c3a720931c4&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=e07cc5009f624eb9912a837c934e68aa => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:46.391290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:46.405341 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 414/834] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:46.429466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:46.552972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669575b2-ea3e-421e-b250-6a8f5a8dddcc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:46.591448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3db4bf-c8a6-4552-b205-999f1fed12bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:46.592540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3db4bf-c8a6-4552-b205-999f1fed12bc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.592540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3db4bf-c8a6-4552-b205-999f1fed12bc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.592737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3db4bf-c8a6-4552-b205-999f1fed12bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.604195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3db4bf-c8a6-4552-b205-999f1fed12bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9c3db4bf-c8a6-4552-b205-999f1fed12bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:46.612367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:46.613308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:46.614153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 421/835] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:12:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:46.619778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33152a-5031-49e0-b2ca-bbdf990bceed None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:46.726196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33152a-5031-49e0-b2ca-bbdf990bceed None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:46.763945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd7013-2b37-4dbd-8571-6ade41e26381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:46.764667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd7013-2b37-4dbd-8571-6ade41e26381 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.764969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd7013-2b37-4dbd-8571-6ade41e26381 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.766291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd7013-2b37-4dbd-8571-6ade41e26381 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.767426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd7013-2b37-4dbd-8571-6ade41e26381 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:46.767749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd7013-2b37-4dbd-8571-6ade41e26381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:46.769175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd7013-2b37-4dbd-8571-6ade41e26381 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:46.780468 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 415/836] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:12:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:46.786125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3db4bf-c8a6-4552-b205-999f1fed12bc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:46.789463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:46.897177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3db4bf-c8a6-4552-b205-999f1fed12bc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:46.933698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:46.934881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.935406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.935886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.947729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:46.949510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:46.958485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b5d88-3072-4ad5-b900-27f6d7313549 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.958766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b5d88-3072-4ad5-b900-27f6d7313549 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.959036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b5d88-3072-4ad5-b900-27f6d7313549 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.960672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 416/837] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:46.966278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a17c7-7ead-450d-ac3e-23ffa85e04a2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:46.966569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a17c7-7ead-450d-ac3e-23ffa85e04a2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:46.966852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a17c7-7ead-450d-ac3e-23ffa85e04a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:46.970102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:46.970962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:46.971896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:46.971896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:46.971896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:46.971896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:46.972774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 422/838] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:46.978622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:46.986839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-201a17c7-7ead-450d-ac3e-23ffa85e04a2 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:46.988126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-201a17c7-7ead-450d-ac3e-23ffa85e04a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:47.038331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:47.038527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:47.067676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 417/839] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:47.086645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d2059-095f-42f7-a295-8fd077da7836 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.090568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a17c7-7ead-450d-ac3e-23ffa85e04a2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.126804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.127500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.127883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.128166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.129161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:47.129478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:47.130912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:47.149444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 423/840] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:47.151060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.200425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a17c7-7ead-450d-ac3e-23ffa85e04a2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.238731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5a57-b1e5-421b-a6a6-656b8bff02a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.239670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5a57-b1e5-421b-a6a6-656b8bff02a6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.240017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5a57-b1e5-421b-a6a6-656b8bff02a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.240363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5a57-b1e5-421b-a6a6-656b8bff02a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.251793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f5a57-b1e5-421b-a6a6-656b8bff02a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e75f5a57-b1e5-421b-a6a6-656b8bff02a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:47.260837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:47.261800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:47.262640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 418/841] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:47.263480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4e1c2-6cec-484e-84c1-83aaf2147abe None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.269289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5a57-b1e5-421b-a6a6-656b8bff02a6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.277941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:47.300950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.301952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.302366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.302709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.310806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:47.312425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:47.317737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:47.332820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:47.333792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:47.334760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:47.342053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 424/842] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:47.343219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.379007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5a57-b1e5-421b-a6a6-656b8bff02a6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.418930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.419733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.420084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.420433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.421249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:47.421632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:47.423504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:47.437249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 419/843] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:47.438194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.459376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145822c6-23d3-409a-ad1f-106ad86fc4fa None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.497417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.498394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.498868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.499326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.500560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:47.501071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:47.502829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:47.527133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 425/844] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:47.528481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.547259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de3f9e-4d0b-4c2e-9c4a-dde9907bf1a5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.549917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:47.565196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:47.588602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.589494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.589842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.590179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.601648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:47.603329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:47.624225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:47.625318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:47.626331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:47.626331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:47.626331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:47.626331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:47.630364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 420/845] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:47.631538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.648275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b163b52-293a-4dd9-9f25-8e0822871f6b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.686802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00c1cd-2971-4796-8217-528c874bd5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.687857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00c1cd-2971-4796-8217-528c874bd5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.688354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00c1cd-2971-4796-8217-528c874bd5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.688867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00c1cd-2971-4796-8217-528c874bd5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.742119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca590d4-5e9d-4212-9543-dbe6c78538c6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.767476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00c1cd-2971-4796-8217-528c874bd5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:47.769447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00c1cd-2971-4796-8217-528c874bd5b2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:47.780792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.781519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.781874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.782223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.783148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:47.783514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:47.784957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:47.798539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 421/846] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:47.803992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.804761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:47.825465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:47.889898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 426/847] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:12:47.890933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00c1cd-2971-4796-8217-528c874bd5b2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.916480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef9929-823e-41ad-a61d-50c882ed456b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:47.954094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:47.955225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:47.955708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:47.956170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:47.964310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:47.966153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:47.982385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:47.983400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 17:12:47.984638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:47.991691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 422/848] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:47.992812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:47.998820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00c1cd-2971-4796-8217-528c874bd5b2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.034479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:48.039841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.041550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.042296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.042970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.055225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.057652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:48.078253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:48.079426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:48.080414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:48.080414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:48.080414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:48.080414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:48.080636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 427/849] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:48.085808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.087003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:48.124535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f5a49-436e-471c-95e4-62170b0009be None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.165568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.166553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.166996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.167446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.168447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:48.169016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.170674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.195292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330cf15-d01c-405c-8fa3-be45e2c001b6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.195674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 423/850] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 17:12:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:48.196925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.232068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7cbc2-a959-45ca-b47a-eab02c2fe864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.232733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7cbc2-a959-45ca-b47a-eab02c2fe864 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.233015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7cbc2-a959-45ca-b47a-eab02c2fe864 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.233295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7cbc2-a959-45ca-b47a-eab02c2fe864 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.234021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7cbc2-a959-45ca-b47a-eab02c2fe864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:48.234570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7cbc2-a959-45ca-b47a-eab02c2fe864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.235978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7cbc2-a959-45ca-b47a-eab02c2fe864 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.251956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 428/851] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:48.251956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cbc80-68bb-49c3-9638-b4c15585101f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.251956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cbc80-68bb-49c3-9638-b4c15585101f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.251956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cbc80-68bb-49c3-9638-b4c15585101f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.251956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 429/852] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:48.253158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cbc80-68bb-49c3-9638-b4c15585101f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.302957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:48.311502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974ab22-f522-44e6-8099-01fa1fdb2555 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.348869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319e5dd-ca08-47e6-a5ad-b0451d89f9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.349524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e5dd-ca08-47e6-a5ad-b0451d89f9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.349806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e5dd-ca08-47e6-a5ad-b0451d89f9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.350081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e5dd-ca08-47e6-a5ad-b0451d89f9b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.352935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:48.360789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cbc80-68bb-49c3-9638-b4c15585101f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.395697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c717b6-49f1-484b-9c0a-fbf0a8fdcc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.396642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c717b6-49f1-484b-9c0a-fbf0a8fdcc26 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.397025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c717b6-49f1-484b-9c0a-fbf0a8fdcc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.397391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c717b6-49f1-484b-9c0a-fbf0a8fdcc26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.405072 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c717b6-49f1-484b-9c0a-fbf0a8fdcc26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.406618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c717b6-49f1-484b-9c0a-fbf0a8fdcc26 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-39c717b6-49f1-484b-9c0a-fbf0a8fdcc26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:48.421777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:48.422892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:48.424049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:48.424049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:48.424049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:48.424049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:48.424049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:48.424049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:48.424049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:48.424049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:48.429672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 430/853] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:48.429672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d79eb-8c86-4b42-b32d-7d9615bcc726 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.429672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d79eb-8c86-4b42-b32d-7d9615bcc726 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.429672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d79eb-8c86-4b42-b32d-7d9615bcc726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.432664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319e5dd-ca08-47e6-a5ad-b0451d89f9b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.434191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319e5dd-ca08-47e6-a5ad-b0451d89f9b7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.444070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f15d79eb-8c86-4b42-b32d-7d9615bcc726 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:48.445336 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f15d79eb-8c86-4b42-b32d-7d9615bcc726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:48.515377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 431/854] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:48.517378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d79eb-8c86-4b42-b32d-7d9615bcc726 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.544382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 424/855] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:12:48.545396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319e5dd-ca08-47e6-a5ad-b0451d89f9b7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.569342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:48.587595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:48.646822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d79eb-8c86-4b42-b32d-7d9615bcc726 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.658544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319e5dd-ca08-47e6-a5ad-b0451d89f9b7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.686594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.687682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.688221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.688505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.689545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:48.690232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.692034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.694656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.695608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.695897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.696184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.697367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.698734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.710713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 432/856] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:48.710713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:48.711000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:48.712076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:48.713578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:48.717559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 425/857] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:48.718479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.812351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:48.818467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:48.819583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730a4c7-cc52-4087-b596-ec8eea4262c1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.822939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ce18e-0b99-461f-8554-b2d941de6331 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:48.852571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57908b-8657-4be3-8cda-cd5b65f47f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.853962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57908b-8657-4be3-8cda-cd5b65f47f91 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.854242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57908b-8657-4be3-8cda-cd5b65f47f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.854517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57908b-8657-4be3-8cda-cd5b65f47f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.856280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:48.856929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.857212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.857488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.858319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=ba50db32ce9f444e919e667305594d4a) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:48.858630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.860351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.865447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57908b-8657-4be3-8cda-cd5b65f47f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:48.866107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 426/858] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/role_assignments?user.id=ba50db32ce9f444e919e667305594d4a&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:48.866866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57908b-8657-4be3-8cda-cd5b65f47f91 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:48.866974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2a57908b-8657-4be3-8cda-cd5b65f47f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:48.890766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:48.892075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:48.896577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:48.896577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:48.896577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:48.896577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:48.896577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 433/859] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:48.896577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbc8e9-ea87-4903-af92-db0b9304f3ef None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:48.896577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbc8e9-ea87-4903-af92-db0b9304f3ef None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:48.896577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbc8e9-ea87-4903-af92-db0b9304f3ef None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:48.897538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 434/860] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:48.898631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbc8e9-ea87-4903-af92-db0b9304f3ef None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:48.989848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d0571-e4f6-4e24-be43-39058adc6e23 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.017274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbc8e9-ea87-4903-af92-db0b9304f3ef None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.029240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce727b8c-0536-4bf1-9571-04323fadc46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.030237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce727b8c-0536-4bf1-9571-04323fadc46f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.030694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce727b8c-0536-4bf1-9571-04323fadc46f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.031145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce727b8c-0536-4bf1-9571-04323fadc46f None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.032082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce727b8c-0536-4bf1-9571-04323fadc46f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:49.032558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce727b8c-0536-4bf1-9571-04323fadc46f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.034101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce727b8c-0536-4bf1-9571-04323fadc46f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.044839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 427/861] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:49.046510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75bdb-caa7-4ab0-a11c-91514ac20de3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.046950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75bdb-caa7-4ab0-a11c-91514ac20de3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.047406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75bdb-caa7-4ab0-a11c-91514ac20de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.056673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.057341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.057625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.057896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.058626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:49.058934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.060358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.067165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-90d75bdb-caa7-4ab0-a11c-91514ac20de3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:49.068374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-90d75bdb-caa7-4ab0-a11c-91514ac20de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:49.074418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 435/862] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:49.075030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:49.083645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:49.092223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:49.148076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 428/863] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:49.148773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d75bdb-caa7-4ab0-a11c-91514ac20de3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:49.191937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6890706-aa81-4f71-8cac-1eaa7f7abc33 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.229570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.230233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.230510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.230779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.231521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:49.231824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.233208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.238270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 436/864] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:49.240452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:49.262983 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d75bdb-caa7-4ab0-a11c-91514ac20de3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.310348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.311828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.312974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.313419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.325394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.328202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.336276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:49.350084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:49.351270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:49.352417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:49.352417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:49.352417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:49.352417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:49.352417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:49.352417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:49.352417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:49.352417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:49.356169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 429/865] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:49.367118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:49.377234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6aba4-95a0-4484-90ed-88e0bfb902ce None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.391844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:49.417285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.418119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.418406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.418684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.419383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.420772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:49.435932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:49.436967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:49.437833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:49.439409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 437/866] 10.222.0.33 () {56 vars in 1132 bytes} [Sat May 28 17:12:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:49.442714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:49.498909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801323c-07e6-4e94-903a-6efbb2ed9206 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.536381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.537449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.537915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.538369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.539602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:49.540172 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.542568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.555702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06eaca-639c-4b55-9863-e0129404e020 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.556863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 430/867] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:49.563517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:49.596332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.596838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.597120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.597401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.598112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:49.598422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.600023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.600918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:49.611303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 438/868] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:12:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:49.611650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:49.666463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:49.702138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b5d71-8db5-47c9-a085-ed23ad59ddce None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.748995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83362cae-3c92-4354-ab34-ff05475a3707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.749670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362cae-3c92-4354-ab34-ff05475a3707 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.749955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362cae-3c92-4354-ab34-ff05475a3707 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.750237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362cae-3c92-4354-ab34-ff05475a3707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.751531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83362cae-3c92-4354-ab34-ff05475a3707 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=c4224f58ecbb423dae16d5b9ae09d5cb) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:49.751854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83362cae-3c92-4354-ab34-ff05475a3707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.753916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83362cae-3c92-4354-ab34-ff05475a3707 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.755537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509f9a0-1d4e-443e-9743-96c5aa37e9a1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:49.767920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 431/869] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:49 2022] GET /identity/v3/role_assignments?user.id=c4224f58ecbb423dae16d5b9ae09d5cb&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=e07cc5009f624eb9912a837c934e68aa => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:49.799050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f10d92-e1a2-42e0-8116-51b52d85c851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:49.800196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10d92-e1a2-42e0-8116-51b52d85c851 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:49.800699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10d92-e1a2-42e0-8116-51b52d85c851 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:49.801189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10d92-e1a2-42e0-8116-51b52d85c851 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:49.802187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f10d92-e1a2-42e0-8116-51b52d85c851 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:49.802721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f10d92-e1a2-42e0-8116-51b52d85c851 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:49.804410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f10d92-e1a2-42e0-8116-51b52d85c851 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:49.811046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 439/870] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:12:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:51.217196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984df89d-e9c4-4108-b24f-b5a959dcac3a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:51.217758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984df89d-e9c4-4108-b24f-b5a959dcac3a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:51.217758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984df89d-e9c4-4108-b24f-b5a959dcac3a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:51.219348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 432/871] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:51.224885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c98fcd-3f84-4245-8562-12cdbadc3c31 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:51.225377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c98fcd-3f84-4245-8562-12cdbadc3c31 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:51.225477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c98fcd-3f84-4245-8562-12cdbadc3c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:51.243041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b3c98fcd-3f84-4245-8562-12cdbadc3c31 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:51.244280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b3c98fcd-3f84-4245-8562-12cdbadc3c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:51.308332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:51.319711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 440/872] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:51.343285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984df89d-e9c4-4108-b24f-b5a959dcac3a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:51.454355 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984df89d-e9c4-4108-b24f-b5a959dcac3a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:51.488161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fc28-b917-43ba-84a3-0f28524f4cad None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:51.488347 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fc28-b917-43ba-84a3-0f28524f4cad None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:51.488639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fc28-b917-43ba-84a3-0f28524f4cad None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:51.491455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 441/873] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:51.492032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e9e10-371c-4cef-8337-7da371e84a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:51.493026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e9e10-371c-4cef-8337-7da371e84a17 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:51.493500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e9e10-371c-4cef-8337-7da371e84a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:51.493921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e9e10-371c-4cef-8337-7da371e84a17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:51.495789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcc261-1a6c-4a52-b312-ebb10bdd92b1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:51.496103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcc261-1a6c-4a52-b312-ebb10bdd92b1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:51.496387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcc261-1a6c-4a52-b312-ebb10bdd92b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:51.503407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:51.508174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e9e10-371c-4cef-8337-7da371e84a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:51.517514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7dbcc261-1a6c-4a52-b312-ebb10bdd92b1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9f4e9e10-371c-4cef-8337-7da371e84a17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:51.517729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:51.518632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:51.519798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 433/874] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 17:12:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:51.522594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7dbcc261-1a6c-4a52-b312-ebb10bdd92b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:51.531276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e9e10-371c-4cef-8337-7da371e84a17 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:51.614434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 442/875] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:51.637060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcc261-1a6c-4a52-b312-ebb10bdd92b1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:51.648388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e9e10-371c-4cef-8337-7da371e84a17 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:51.686409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:51.687117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:51.687509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:51.687793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:51.688609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:51.688919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:51.690335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:51.705719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 434/876] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 17:12:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:51.706747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:51.750500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcc261-1a6c-4a52-b312-ebb10bdd92b1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:51.773170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:51.785935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f30af-c10e-4cd0-8433-71837aabd69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:51.786883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f30af-c10e-4cd0-8433-71837aabd69f None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:51.787312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f30af-c10e-4cd0-8433-71837aabd69f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:51.787757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f30af-c10e-4cd0-8433-71837aabd69f None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:51.788643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f30af-c10e-4cd0-8433-71837aabd69f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:51.789642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:51.790946 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f30af-c10e-4cd0-8433-71837aabd69f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:51.806992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 443/877] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:12:51 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:12:51.846652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f9d7-31a5-47f6-87ac-c14c6f1ce923 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:51.895558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:51.897173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:51.897684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:51.898158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:51.911418 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:51.913249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:51.933345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:51.934317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:51.935249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:51.935249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:51.935249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:51.935249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:51.937705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 435/878] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:51.943632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f30af-c10e-4cd0-8433-71837aabd69f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:52.067231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f30af-c10e-4cd0-8433-71837aabd69f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:52.102658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258149-b6cf-485b-9979-e43e2fca91ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:52.103552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258149-b6cf-485b-9979-e43e2fca91ae None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.103920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258149-b6cf-485b-9979-e43e2fca91ae None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.104200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258149-b6cf-485b-9979-e43e2fca91ae None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.104931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258149-b6cf-485b-9979-e43e2fca91ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:52.105240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258149-b6cf-485b-9979-e43e2fca91ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:52.106627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258149-b6cf-485b-9979-e43e2fca91ae None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:52.120403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 444/879] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:52.125851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:52.160147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded39145-94c1-43b2-ba15-d142fcd8162b None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.160480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded39145-94c1-43b2-ba15-d142fcd8162b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.160694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded39145-94c1-43b2-ba15-d142fcd8162b None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.162321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 445/880] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:12:52.168659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca1d39-90e3-4141-ac54-20bf388c1be6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.168936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca1d39-90e3-4141-ac54-20bf388c1be6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.169218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca1d39-90e3-4141-ac54-20bf388c1be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.190637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-45ca1d39-90e3-4141-ac54-20bf388c1be6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:52.192026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-45ca1d39-90e3-4141-ac54-20bf388c1be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:52.235116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:52.245655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c2a9-44af-42b0-b4c6-f196f0d84e9b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:52.256665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:52.273682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 446/881] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:52.275645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675e2c-2beb-4151-8ef2-5fed7b3336a6 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.276099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675e2c-2beb-4151-8ef2-5fed7b3336a6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.276680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675e2c-2beb-4151-8ef2-5fed7b3336a6 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.278713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 447/882] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:52.284126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58934c-42f5-4150-b8e0-516363d9a245 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.284509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58934c-42f5-4150-b8e0-516363d9a245 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.284877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58934c-42f5-4150-b8e0-516363d9a245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.286074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:52.287052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.287534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.287970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.297098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:52.298813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:52.312599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3a58934c-42f5-4150-b8e0-516363d9a245 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:52.313886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3a58934c-42f5-4150-b8e0-516363d9a245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:52.315459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:52.316420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:52.317420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:52.322944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 436/883] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:12:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:52.322944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:52.395631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 448/884] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:52.396712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58934c-42f5-4150-b8e0-516363d9a245 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:52.429437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d98cd9-53ec-44d2-a27d-9462e5797ddb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:52.468038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:52.468851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.469132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.469406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.470064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:52.471480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:52.484904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:52.486052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:52.486898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 437/885] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:12:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:52.487750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:52.521195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58934c-42f5-4150-b8e0-516363d9a245 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:52.523249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:52.529776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:52.560319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:52.561034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.561315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.561594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.562434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:52.562742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:52.564195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:52.587149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 449/886] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 17:12:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:52.587149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:52.607818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea687542-9d38-4540-bb54-86a65e2e1192 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:52.647433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b3893-2496-4856-86e2-e7e437472c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:52.648096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b3893-2496-4856-86e2-e7e437472c9d None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.648376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b3893-2496-4856-86e2-e7e437472c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.648650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b3893-2496-4856-86e2-e7e437472c9d None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.649507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b3893-2496-4856-86e2-e7e437472c9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:52.652111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b3893-2496-4856-86e2-e7e437472c9d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:52.668631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 438/887] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:12:52 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:52.669438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b3893-2496-4856-86e2-e7e437472c9d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:52.729699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3d161-cb87-41fd-b2cf-87e8faeb98e2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:52.777621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:52.778455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.778860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.779249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.780097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:52.780511 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:52.782292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:52.792355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 450/888] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:12:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:52.794214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:52.829999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:52.836440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b3893-2496-4856-86e2-e7e437472c9d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:52.875836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:52.896679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008e85c-dc47-48d3-bf47-b437d77878ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:52.897486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008e85c-dc47-48d3-bf47-b437d77878ba None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.898012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008e85c-dc47-48d3-bf47-b437d77878ba None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.898516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008e85c-dc47-48d3-bf47-b437d77878ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.942874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc3088-6416-4f86-8664-337b41724776 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:52.989292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:52.990304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:52.990840 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:52.991327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:52.992231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:52.992728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:52.995005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:53.000074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 451/889] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:12:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:53.002578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008e85c-dc47-48d3-bf47-b437d77878ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:53.004310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008e85c-dc47-48d3-bf47-b437d77878ba None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:53.111811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:53.121262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 439/890] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:12:52 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:12:53.126832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:53.224172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:53.244509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604f9fc-e31c-4356-a27d-564a80ce2e24 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:53.281631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:53.282793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:53.283272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:53.283778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:53.294529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:53.296292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:53.314071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:53.315000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:53.315911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:53.315911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:53.315911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:53.315911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:53.318107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 452/891] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:53.324448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008e85c-dc47-48d3-bf47-b437d77878ba None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:53.428425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008e85c-dc47-48d3-bf47-b437d77878ba None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:53.469563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:53.470236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:53.470519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:53.470801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:53.471563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:53.471874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:53.473249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:53.485794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 440/892] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:53.491673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:53.593861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6641e-14fa-4e24-bd9d-14e9ff8ad36b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:53.630415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:53.631622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:53.632162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:53.632630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:53.635874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:53.644578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:53.646330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:53.661916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:53.662858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:53.663816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:53.663816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:53.663816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:53.663816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:53.663816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:53.663816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:53.663816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:53.663816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:53.667036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 453/893] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:53.672934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:53.738358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:53.787188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e7f62-746a-4a5c-8802-e81388efec71 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:53.825811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91629935-50ce-4a41-9ac3-879306cad6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:53.826488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629935-50ce-4a41-9ac3-879306cad6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:53.826773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629935-50ce-4a41-9ac3-879306cad6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:53.827070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629935-50ce-4a41-9ac3-879306cad6a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:53.827975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91629935-50ce-4a41-9ac3-879306cad6a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:53.828302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91629935-50ce-4a41-9ac3-879306cad6a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:53.829917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91629935-50ce-4a41-9ac3-879306cad6a7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:53.843710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 441/894] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:53.849520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:53.958022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e3271-f9d9-4519-8be6-ee60932ede00 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:53.995166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23210ecc-f5b2-48db-aa44-53f6761ff66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:53.995774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23210ecc-f5b2-48db-aa44-53f6761ff66b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:53.996060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23210ecc-f5b2-48db-aa44-53f6761ff66b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:53.996341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23210ecc-f5b2-48db-aa44-53f6761ff66b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:53.997174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23210ecc-f5b2-48db-aa44-53f6761ff66b None admin] RBAC: Adding query filter params (role.id=883cfd8ec8cd40b583eea092a23386af, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=ba50db32ce9f444e919e667305594d4a) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:53.997489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23210ecc-f5b2-48db-aa44-53f6761ff66b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:53.998935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23210ecc-f5b2-48db-aa44-53f6761ff66b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:54.004960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 454/895] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:12:53 2022] GET /identity/v3/role_assignments?user.id=ba50db32ce9f444e919e667305594d4a&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=883cfd8ec8cd40b583eea092a23386af => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:54.227074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150e3c1-adc2-4db4-a5f1-0a0b72c0bc65 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:54.227492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150e3c1-adc2-4db4-a5f1-0a0b72c0bc65 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:54.227744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150e3c1-adc2-4db4-a5f1-0a0b72c0bc65 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:54.229462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 442/896] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:54.235141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6544d-3b48-46b1-aa11-b507abd00736 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:54.235465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6544d-3b48-46b1-aa11-b507abd00736 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:54.235750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6544d-3b48-46b1-aa11-b507abd00736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:54.253103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-afb6544d-3b48-46b1-aa11-b507abd00736 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:54.254218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-afb6544d-3b48-46b1-aa11-b507abd00736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:54.311135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:54.323100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 455/897] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:54.345827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150e3c1-adc2-4db4-a5f1-0a0b72c0bc65 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:54.455020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150e3c1-adc2-4db4-a5f1-0a0b72c0bc65 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:54.470130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:54.491518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:54.492323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:54.492610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:54.492887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:54.494223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:54.495692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:54.507838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:54.509599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:54.510622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 443/898] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:54.515768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6544d-3b48-46b1-aa11-b507abd00736 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:54.624248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6544d-3b48-46b1-aa11-b507abd00736 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:54.659307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:54.660007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:54.660290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:54.660566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:54.661287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:54.661592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:54.662965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:54.669028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 456/899] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:54.674322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:54.787133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f10ec-5072-4cd1-ace5-8e2d330e60b3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:54.824301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046827e-283d-456d-a867-f288e56b534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:54.825381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046827e-283d-456d-a867-f288e56b534f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:54.825872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046827e-283d-456d-a867-f288e56b534f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:54.826362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046827e-283d-456d-a867-f288e56b534f None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:54.827293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046827e-283d-456d-a867-f288e56b534f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:54.827845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046827e-283d-456d-a867-f288e56b534f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:54.829437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046827e-283d-456d-a867-f288e56b534f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:54.835459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 444/900] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:54.841514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:54.900361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:54.948096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6160f-3e5d-43a2-96d5-b8dde766eade None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:54.986364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3425ebe-fb7e-44b7-9f93-4054d9d6dd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:54.987052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3425ebe-fb7e-44b7-9f93-4054d9d6dd72 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:54.987353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3425ebe-fb7e-44b7-9f93-4054d9d6dd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:54.987642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3425ebe-fb7e-44b7-9f93-4054d9d6dd72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:54.988462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3425ebe-fb7e-44b7-9f93-4054d9d6dd72 None admin] RBAC: Adding query filter params (interface=public, service_id=0be5a73b48bf48f69de7ad2f1a1fab83, region_id=RegionOne) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:54.988780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3425ebe-fb7e-44b7-9f93-4054d9d6dd72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:54.990209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3425ebe-fb7e-44b7-9f93-4054d9d6dd72 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:54.999921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 457/901] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 17:12:54 2022] GET /identity/v3/endpoints?service_id=0be5a73b48bf48f69de7ad2f1a1fab83&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:55.005565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046827e-283d-456d-a867-f288e56b534f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:55.052698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:55.116024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046827e-283d-456d-a867-f288e56b534f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:55.154835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008e2b2-5db2-45bf-9e63-e47aaae0c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:55.155917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008e2b2-5db2-45bf-9e63-e47aaae0c504 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.156410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008e2b2-5db2-45bf-9e63-e47aaae0c504 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.156890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008e2b2-5db2-45bf-9e63-e47aaae0c504 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.157781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008e2b2-5db2-45bf-9e63-e47aaae0c504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:55.159409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008e2b2-5db2-45bf-9e63-e47aaae0c504 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:55.166047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 445/902] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:12:55 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:12:55.175043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee6438-8e57-4c6f-b9c4-2bba28e7c953 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.175397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee6438-8e57-4c6f-b9c4-2bba28e7c953 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.175643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee6438-8e57-4c6f-b9c4-2bba28e7c953 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.178622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 458/903] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:55.183964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd295d-ba94-411f-9ea2-dbfbff4f78cd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.184610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd295d-ba94-411f-9ea2-dbfbff4f78cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.185095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd295d-ba94-411f-9ea2-dbfbff4f78cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.204237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9ebd295d-ba94-411f-9ea2-dbfbff4f78cd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:55.205969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9ebd295d-ba94-411f-9ea2-dbfbff4f78cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:55.288572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 446/904] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:55.312158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee6438-8e57-4c6f-b9c4-2bba28e7c953 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:55.414382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903da7e-d518-49f2-a345-242f4ceed1fe None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.415425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903da7e-d518-49f2-a345-242f4ceed1fe None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.416129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903da7e-d518-49f2-a345-242f4ceed1fe None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.417970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 447/905] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:12:55.423224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107ddbc-63b5-4390-a8df-2ea107d6680d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.423799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107ddbc-63b5-4390-a8df-2ea107d6680d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.424300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107ddbc-63b5-4390-a8df-2ea107d6680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.425347 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee6438-8e57-4c6f-b9c4-2bba28e7c953 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:55.441991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c107ddbc-63b5-4390-a8df-2ea107d6680d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:55.443552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c107ddbc-63b5-4390-a8df-2ea107d6680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:55.444179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:55.463273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:55.463891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.464171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.464443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.465151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:55.466516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:55.480057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:55.481021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:55.481871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:55.483939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:55.486543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 459/906] 10.222.0.33 () {56 vars in 1132 bytes} [Sat May 28 17:12:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:55.487869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:55.527879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 448/907] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:55.551464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107ddbc-63b5-4390-a8df-2ea107d6680d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:55.592108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a36d3-9584-4fb4-af63-47cbb55ec85a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:55.626574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:55.627241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.627573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.628672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.629177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:55.629707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:55.631857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:55.642575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 460/908] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:12:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:55.643825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:55.661813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107ddbc-63b5-4390-a8df-2ea107d6680d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:55.700287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c81bc-86c5-4e38-82a0-57218b968838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:55.701928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c81bc-86c5-4e38-82a0-57218b968838 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.702725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c81bc-86c5-4e38-82a0-57218b968838 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.703503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c81bc-86c5-4e38-82a0-57218b968838 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.705007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c81bc-86c5-4e38-82a0-57218b968838 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:55.707628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c81bc-86c5-4e38-82a0-57218b968838 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:55.727409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:55.743864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 449/909] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:12:55 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:12:55.756180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bafc49-b0a5-4043-b215-b710e7a680e2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:55.804188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:55.804922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.805207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.805482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.806253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:55.806564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:55.808107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:55.814386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 461/910] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:12:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:55.820079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c81bc-86c5-4e38-82a0-57218b968838 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:55.856216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:55.939503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c81bc-86c5-4e38-82a0-57218b968838 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:55.975531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2b318-7e5f-4b1e-a349-344b5700a27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:55.976197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2b318-7e5f-4b1e-a349-344b5700a27b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:55.976481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2b318-7e5f-4b1e-a349-344b5700a27b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:55.976755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2b318-7e5f-4b1e-a349-344b5700a27b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:55.977548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2b318-7e5f-4b1e-a349-344b5700a27b None admin] RBAC: Adding query filter params (interface=public, service_id=172d624cd077440ebe4a7b72b830378e, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:55.978450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2b318-7e5f-4b1e-a349-344b5700a27b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:55.979853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2b318-7e5f-4b1e-a349-344b5700a27b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:55.986276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 450/911] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 17:12:55 2022] GET /identity/v3/endpoints?service_id=172d624cd077440ebe4a7b72b830378e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:55.991714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:56.106844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51706197-8985-4a80-a5a6-f415d824d9c8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:56.114819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:56.144709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6510f37-c41b-4e02-baea-a9434926c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:56.145706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6510f37-c41b-4e02-baea-a9434926c331 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:56.146171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6510f37-c41b-4e02-baea-a9434926c331 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:56.146621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6510f37-c41b-4e02-baea-a9434926c331 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:56.147515 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6510f37-c41b-4e02-baea-a9434926c331 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:56.149056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6510f37-c41b-4e02-baea-a9434926c331 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:56.159430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 462/912] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:12:55 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:12:56.248042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735560a2-8970-45d2-b463-6c984843bfd3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:56.248690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735560a2-8970-45d2-b463-6c984843bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:56.249170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735560a2-8970-45d2-b463-6c984843bfd3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:56.251101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 451/913] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:56.256438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71763dde-bfc5-4386-a93a-3d16386cc24a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:56.257070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71763dde-bfc5-4386-a93a-3d16386cc24a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:56.257553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71763dde-bfc5-4386-a93a-3d16386cc24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:56.276435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-71763dde-bfc5-4386-a93a-3d16386cc24a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:56.278015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-71763dde-bfc5-4386-a93a-3d16386cc24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:56.356068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 463/914] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:56.379010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735560a2-8970-45d2-b463-6c984843bfd3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:56.485592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:56.497040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735560a2-8970-45d2-b463-6c984843bfd3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:56.533208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1648f1-0736-4809-b817-574d9343d052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:56.534031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1648f1-0736-4809-b817-574d9343d052 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:56.534316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1648f1-0736-4809-b817-574d9343d052 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:56.534657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1648f1-0736-4809-b817-574d9343d052 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:56.547906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1648f1-0736-4809-b817-574d9343d052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8b1648f1-0736-4809-b817-574d9343d052 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:12:56.557607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:56.558566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:56.560043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 452/915] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 17:12:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:56.568504 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71763dde-bfc5-4386-a93a-3d16386cc24a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:56.669558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:56.675916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71763dde-bfc5-4386-a93a-3d16386cc24a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:56.713810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:56.714499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:56.714781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:56.715059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:56.715859 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:56.716171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:56.717556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:56.726514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 464/916] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 17:12:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:56.732533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1648f1-0736-4809-b817-574d9343d052 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:56.838310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1648f1-0736-4809-b817-574d9343d052 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:56.881452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:56.882264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:56.882548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:56.882834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:56.896864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:56.898341 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:56.916864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:56.918969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:56.921181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:56.921181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:56.921181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:56.921181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:56.921181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 453/917] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:56.923756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:57.038664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750c4b8-31b7-48d9-9448-5ef6816e706f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:57.075106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:57.075807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:57.076090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:57.076363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:57.077097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:57.077407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:57.078791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:57.092122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 465/918] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:57.100427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:57.103374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:57.210680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f194d-6226-4956-bf4a-74f93c2a6086 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:57.246402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:57.247209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:57.247523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:57.247803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:57.256263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:57.257708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:12:57.277854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:57.278889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:57.279845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 454/919] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:12:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:57.286682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:57.316369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:57.394997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e44e63-1dc5-45ac-b0a0-2fb05f3e373b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:57.436941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be39346-5a9e-44aa-ba82-21ba5b9c161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:57.437621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be39346-5a9e-44aa-ba82-21ba5b9c161f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:57.437905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be39346-5a9e-44aa-ba82-21ba5b9c161f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:57.438184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be39346-5a9e-44aa-ba82-21ba5b9c161f None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:57.439035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be39346-5a9e-44aa-ba82-21ba5b9c161f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:57.439367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be39346-5a9e-44aa-ba82-21ba5b9c161f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:57.440809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be39346-5a9e-44aa-ba82-21ba5b9c161f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:57.459733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 466/920] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 17:12:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:57.468494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:57.538503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:57.586594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04dfc2-8c6e-4356-a78c-f4fb7c107ddd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:57.606834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f363780-abeb-45d6-a688-1ded8fedee98 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:57.607129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f363780-abeb-45d6-a688-1ded8fedee98 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:57.607427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f363780-abeb-45d6-a688-1ded8fedee98 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:57.609067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 467/921] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:12:57.614206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b0904-197a-43e1-96e5-8756bbd0631a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:57.614498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b0904-197a-43e1-96e5-8756bbd0631a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:57.614780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b0904-197a-43e1-96e5-8756bbd0631a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:57.626007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:57.626854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:57.627130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:57.627633 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:57.633685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3d1b0904-197a-43e1-96e5-8756bbd0631a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:57.635683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3d1b0904-197a-43e1-96e5-8756bbd0631a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:57.640232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:57.641740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:12:57.665101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:57.666592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:57.667958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:12:57.667958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:12:57.667958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:12:57.667958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:57.671834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 455/922] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:57.672667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:57.743621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 468/923] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:57.758254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b0904-197a-43e1-96e5-8756bbd0631a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:57.770941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:57.791782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4f7e9-6e99-43ea-a9ee-d266a3e0c14d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:57.807747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:57.830999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:57.832008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:57.832445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:57.832859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:57.833825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:57.834523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:57.836257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:57.854430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 456/924] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:57.855730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:57.870513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b0904-197a-43e1-96e5-8756bbd0631a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:57.911467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:57.912300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:57.912586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:57.912865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:57.913550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:57.915559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:57.928741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:57.929837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 17:12:57.930741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:57.935603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 469/925] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:57.936481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:57.975248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79aec8-d82d-4fd3-8bf0-b79912d936d7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.013566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.014835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.015313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.015832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.019689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:58.024130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.026451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:12:58.042671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:58.043672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:58.044762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:12:58.044762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:12:58.044762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:12:58.044762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:12:58.044762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:12:58.044762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:12:58.044762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:12:58.044762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:58.051000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08233e-aaa0-4b88-9502-d1e2777c3462 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.051562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 457/926] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:58.052561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:58.087715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481f627-e697-44e4-863d-9c3ae23b6739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.088449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481f627-e697-44e4-863d-9c3ae23b6739 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.088731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481f627-e697-44e4-863d-9c3ae23b6739 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.089268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481f627-e697-44e4-863d-9c3ae23b6739 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.090055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481f627-e697-44e4-863d-9c3ae23b6739 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:58.090374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481f627-e697-44e4-863d-9c3ae23b6739 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.090925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:58.093427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481f627-e697-44e4-863d-9c3ae23b6739 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.100121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 470/927] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:58.101566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c893c-98e9-4f86-85b5-85998684e406 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.101761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c893c-98e9-4f86-85b5-85998684e406 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.101914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c893c-98e9-4f86-85b5-85998684e406 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.103381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 471/928] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:58.111274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f72e-fc1d-4225-8f3b-73583942f0a9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.111774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f72e-fc1d-4225-8f3b-73583942f0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.112218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f72e-fc1d-4225-8f3b-73583942f0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.131207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3609f72e-fc1d-4225-8f3b-73583942f0a9 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:58.132651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3609f72e-fc1d-4225-8f3b-73583942f0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:58.159950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2ee98-b3e0-4e95-88e2-72b9a0a00f16 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.196432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.197678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.197958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.198234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.199014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:58.199595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.200755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.207147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 472/929] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:12:58.212302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609f72e-fc1d-4225-8f3b-73583942f0a9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:58.217765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 458/930] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:12:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:58.218722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:58.295705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:58.318693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:58.327452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a12759-4d57-41b7-b3c5-7004f6c9e2ca None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.330671 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609f72e-fc1d-4225-8f3b-73583942f0a9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.364484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.365146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.365423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.365696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.366398 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:58.366700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.367194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b506d-aa33-45e7-8af9-5c336a0b9f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.368106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.368544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b506d-aa33-45e7-8af9-5c336a0b9f14 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.369005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b506d-aa33-45e7-8af9-5c336a0b9f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.369548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b506d-aa33-45e7-8af9-5c336a0b9f14 None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/ba50db32ce9f444e919e667305594d4a/roles/883cfd8ec8cd40b583eea092a23386af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.372202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b506d-aa33-45e7-8af9-5c336a0b9f14 None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=ba50db32ce9f444e919e667305594d4a, role_id=883cfd8ec8cd40b583eea092a23386af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.373129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 473/931] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 17:12:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:58.373914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:58.374272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b506d-aa33-45e7-8af9-5c336a0b9f14 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.394298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 459/932] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:12:58 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/ba50db32ce9f444e919e667305594d4a/roles/883cfd8ec8cd40b583eea092a23386af => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:12:58.395507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b506d-aa33-45e7-8af9-5c336a0b9f14 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:58.517894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51702e74-ba67-458b-b985-8226540d0913 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.528235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b506d-aa33-45e7-8af9-5c336a0b9f14 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.562332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:58.568957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.569532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcb70f-8c5e-4c8a-b9f2-89a1b504570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.570010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.570092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.570318 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.570986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.571524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb70f-8c5e-4c8a-b9f2-89a1b504570c None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.572034 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb70f-8c5e-4c8a-b9f2-89a1b504570c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.572464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.572700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb70f-8c5e-4c8a-b9f2-89a1b504570c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.573655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcb70f-8c5e-4c8a-b9f2-89a1b504570c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.575259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcb70f-8c5e-4c8a-b9f2-89a1b504570c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:58.586734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:58.587915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:12:58.588745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:12:58.589485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 474/933] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:12:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:12:58.595031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:58.601354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 460/934] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:12:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:12:58.604898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6aea3-acae-41c6-8d13-c702a535c739 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.605446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6aea3-acae-41c6-8d13-c702a535c739 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.605887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6aea3-acae-41c6-8d13-c702a535c739 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.607821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 461/935] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:12:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:12:58.614008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106abfb-7d81-4add-a8bb-ddedbfdd5058 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.614646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106abfb-7d81-4add-a8bb-ddedbfdd5058 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.615118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106abfb-7d81-4add-a8bb-ddedbfdd5058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.638228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2106abfb-7d81-4add-a8bb-ddedbfdd5058 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:12:58.640215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2106abfb-7d81-4add-a8bb-ddedbfdd5058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:12:58.658744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:58.754230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cfdaa-06e8-4c33-bb44-be503cd26912 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.759835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 462/936] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:12:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:12:58.783010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106abfb-7d81-4add-a8bb-ddedbfdd5058 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:58.801245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.802458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.802888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.803594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.804956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:58.805438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.807312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.815236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 475/937] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:12:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:58.820390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:58.901544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:58.917921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106abfb-7d81-4add-a8bb-ddedbfdd5058 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.943693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7e644-d7c6-464a-a22b-0d1d326eae00 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:58.963856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.964683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.964962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.965446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.966214 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.968417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.972841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:58.987212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:12:58.987593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:12:58.988486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 17:12:58.989608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:12:58.990419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:58.990419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:58.990419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:58.990606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 463/938] 10.222.0.33 () {56 vars in 1132 bytes} [Sat May 28 17:12:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:12:58.990681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:58.990681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:58.996594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:58.997597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:59.002612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 476/939] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:12:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:12:59.012519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:59.121249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd12c9b-46dc-4b42-90ba-79c09dc43044 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:59.136567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455461f-775e-4910-8a47-4f9f377f5626 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:59.163962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:59.164200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:59.165979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:59.166378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:59.166758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:59.168026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:59.168336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:59.170346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:59.177256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 464/940] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:12:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:59.178958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:59.179891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:59.180425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:59.180958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:59.182381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] RBAC: Adding query filter params (interface=public, service_id=4c051b0911b94807a637a128f045d131, region_id=RegionOne) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:59.182701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:59.183712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:59.184108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:59.194144 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 477/941] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 17:12:59 2022] GET /identity/v3/endpoints?service_id=4c051b0911b94807a637a128f045d131&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:59.197979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:59.263771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:59.305983 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166a167-a359-44a6-9057-699077006ed7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:59.319471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d7e2d-b511-454c-be23-08b37df43a70 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:59.342533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:59.343585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:59.344078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:59.344555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:59.345479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:12:59.345986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:59.347772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:59.353517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 465/942] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:12:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:12:59.362472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:12:59.362605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b4300-097c-40fa-8ad7-8a048ded6fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:59.362605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b4300-097c-40fa-8ad7-8a048ded6fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:59.362605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b4300-097c-40fa-8ad7-8a048ded6fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:59.362860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b4300-097c-40fa-8ad7-8a048ded6fb6 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:59.364937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b4300-097c-40fa-8ad7-8a048ded6fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:59.365051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b4300-097c-40fa-8ad7-8a048ded6fb6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:59.372331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 478/943] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:12:59 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:12:59.470946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a42268-5079-4df0-840c-cae614c83e4d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:12:59.500937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:12:59.509709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2e5f9-c20c-404e-8c42-aa62842bd555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:12:59.510482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2e5f9-c20c-404e-8c42-aa62842bd555 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:12:59.510777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2e5f9-c20c-404e-8c42-aa62842bd555 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:12:59.511057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2e5f9-c20c-404e-8c42-aa62842bd555 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:12:59.511861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2e5f9-c20c-404e-8c42-aa62842bd555 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:12:59.513256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2e5f9-c20c-404e-8c42-aa62842bd555 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:12:59.539207 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 466/944] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:12:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:13:00.739640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7cb02-7353-4f49-aa45-3aac3877a483 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:00.740120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7cb02-7353-4f49-aa45-3aac3877a483 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:00.740199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7cb02-7353-4f49-aa45-3aac3877a483 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:00.741787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 479/945] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:00.747309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5d7b-b885-4a02-9eef-496c3bc16e1c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:00.747757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5d7b-b885-4a02-9eef-496c3bc16e1c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:00.747958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5d7b-b885-4a02-9eef-496c3bc16e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:00.764921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a73b5d7b-b885-4a02-9eef-496c3bc16e1c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:00.766121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a73b5d7b-b885-4a02-9eef-496c3bc16e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:00.847796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 467/946] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:00.872054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7cb02-7353-4f49-aa45-3aac3877a483 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:00.933413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:00.984866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7cb02-7353-4f49-aa45-3aac3877a483 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:00.992260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c18b77-1d0a-4e04-844b-17f649461b7c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:00.992569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c18b77-1d0a-4e04-844b-17f649461b7c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:00.993200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c18b77-1d0a-4e04-844b-17f649461b7c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:00.995158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 468/947] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:13:01.000014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ce243-8f13-4b08-b572-83688de4ccfd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.000300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ce243-8f13-4b08-b572-83688de4ccfd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.000703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ce243-8f13-4b08-b572-83688de4ccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.020036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-037ce243-8f13-4b08-b572-83688de4ccfd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:01.021208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-037ce243-8f13-4b08-b572-83688de4ccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:01.023156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c4f45-b21e-459c-a7af-ef26cbc3078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.024235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4f45-b21e-459c-a7af-ef26cbc3078a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.024747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4f45-b21e-459c-a7af-ef26cbc3078a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.025237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4f45-b21e-459c-a7af-ef26cbc3078a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.036756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c4f45-b21e-459c-a7af-ef26cbc3078a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-640c4f45-b21e-459c-a7af-ef26cbc3078a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:01.045342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:01.046279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:13:01.047117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:01.049844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 480/948] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 17:13:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:01.055255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c4f45-b21e-459c-a7af-ef26cbc3078a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.107422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 469/949] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:01.131638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ce243-8f13-4b08-b572-83688de4ccfd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.171580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c4f45-b21e-459c-a7af-ef26cbc3078a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:01.185915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:01.191804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:01.209652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.210391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.210669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.210938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.211780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:01.212641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:01.214230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:01.227692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 481/950] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 17:13:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:01.229083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.235619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ce243-8f13-4b08-b572-83688de4ccfd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:01.270784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.271483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.271762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.272036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.272759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:01.274191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:01.276005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING keystone.server.flask.common [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 28 17:13:01.294945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.sql.core [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 17:13:01.294945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 17:13:01.294945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [parameters: {'id': 'ecd16a16dc3e4ea680a5cf188caaa270', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 28 17:13:01.294945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80261) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ecd16a16dc3e4ea680a5cf188caaa270.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ecd16a16dc3e4ea680a5cf188caaa270. May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 17:13:01.304335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 28 17:13:01.305167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return project_ref.to_dict() May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application yield resource May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.rollback() May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.force_reraise() May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise self.value May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application session.commit() May 28 17:13:01.306740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._prepare_impl() May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.flush() May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._flush(objects) May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application compat.raise_( May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application flush_context.execute() May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 17:13:01.307820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rec.execute(self) May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application c = connection._execute_20( May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 17:13:01.309013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 17:13:01.310016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [parameters: {'id': 'ecd16a16dc3e4ea680a5cf188caaa270', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:01.311130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:01.319959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.Conflict( May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ecd16a16dc3e4ea680a5cf188caaa270. May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 470/951] 10.222.0.33 () {60 vars in 1140 bytes} [Sat May 28 17:13:01 2022] POST /identity/v3/projects => generated 262 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:01.322087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.342183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8ad41-5c1e-4eaf-bcf5-fabbb042c33e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:01.378642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.379487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.379768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.380041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.392249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:01.393762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:01.411423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:01.413742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:01.416510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:01.416510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:01.416510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:01.416510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:01.423370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 482/952] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:01.423370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.423752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:01.432495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875ff0a-87c0-4c5d-b3ea-7c250706a914 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:01.448457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:01.471709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.472638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.472922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.473201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.482286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:01.484142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:13:01.501075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:01.503210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:01.505482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 471/953] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:13:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:01.515790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.537061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842f8fd-be62-4383-9a70-8892893b56e3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:01.574440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.575090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.575384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.575658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.576389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:01.576693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:01.578055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:01.591606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 483/954] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:01.596755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.655115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106cb7e-4e02-4b08-8499-99f3edc10685 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:01.689536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:01.694822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce478e9b-8cbd-49fc-88fb-4278484955f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.695529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce478e9b-8cbd-49fc-88fb-4278484955f2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.695815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce478e9b-8cbd-49fc-88fb-4278484955f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.696095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce478e9b-8cbd-49fc-88fb-4278484955f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.697492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce478e9b-8cbd-49fc-88fb-4278484955f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:01.697799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce478e9b-8cbd-49fc-88fb-4278484955f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:01.699232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce478e9b-8cbd-49fc-88fb-4278484955f2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:01.706024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda69e0-4b48-4f0a-a273-46b236444af5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:01.726256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 472/955] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:13:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:01.727971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6593-f73e-427f-8bbf-85a78438c5f3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.728370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6593-f73e-427f-8bbf-85a78438c5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.728743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6593-f73e-427f-8bbf-85a78438c5f3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.731503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 473/956] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:01.736816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a83125-dcf4-4bb2-8f90-32a581a55c0d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.737440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a83125-dcf4-4bb2-8f90-32a581a55c0d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.737898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a83125-dcf4-4bb2-8f90-32a581a55c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.757322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-03a83125-dcf4-4bb2-8f90-32a581a55c0d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:01.759010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.759992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.760331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.760698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.761275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-03a83125-dcf4-4bb2-8f90-32a581a55c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:01.769680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:01.771434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:01.775188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:13:01.793441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:01.794479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:01.795941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 484/957] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:13:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:01.800983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.844092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 474/958] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:01.867650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a83125-dcf4-4bb2-8f90-32a581a55c0d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:01.940159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b03cb4-3b1c-443e-9d71-ed71b001543b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:01.983109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe7b6a-dd57-4894-8126-c2b073f1988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:01.983850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe7b6a-dd57-4894-8126-c2b073f1988e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:01.984129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe7b6a-dd57-4894-8126-c2b073f1988e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:01.984403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe7b6a-dd57-4894-8126-c2b073f1988e None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:01.985499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe7b6a-dd57-4894-8126-c2b073f1988e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:01.985802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe7b6a-dd57-4894-8126-c2b073f1988e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:01.987230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe7b6a-dd57-4894-8126-c2b073f1988e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:01.994786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a83125-dcf4-4bb2-8f90-32a581a55c0d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.010256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 485/959] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 17:13:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:02.011649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1b43a-fb04-4ea4-9b8f-1eb1047a3b8c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.012045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1b43a-fb04-4ea4-9b8f-1eb1047a3b8c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.012322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1b43a-fb04-4ea4-9b8f-1eb1047a3b8c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.013572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 486/960] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:02.019099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8201a52-1d66-44a7-a49f-c432be5aae29 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.019406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8201a52-1d66-44a7-a49f-c432be5aae29 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.019687 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8201a52-1d66-44a7-a49f-c432be5aae29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.026803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:02.037242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c8201a52-1d66-44a7-a49f-c432be5aae29 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:02.037745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.038417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c8201a52-1d66-44a7-a49f-c432be5aae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:02.038560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.038845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.039115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.039804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.042092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:02.054354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:02.055558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:02.057510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 475/961] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:13:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:02.058321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.074869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:02.119442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 487/962] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:02.120613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8201a52-1d66-44a7-a49f-c432be5aae29 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.177227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3734-3e3e-4c16-ab75-901323474215 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.215580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1e348-f74f-4051-b364-38794a9442ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.216232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1e348-f74f-4051-b364-38794a9442ca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.216512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1e348-f74f-4051-b364-38794a9442ca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.217038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1e348-f74f-4051-b364-38794a9442ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.231596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8201a52-1d66-44a7-a49f-c432be5aae29 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.267713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.268295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.268573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.268866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.269550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:02.269854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.271215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.272425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:02.282304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 488/963] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:02.283122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.297165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1e348-f74f-4051-b364-38794a9442ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.298604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1e348-f74f-4051-b364-38794a9442ca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.311466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:02.396160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6715e83-b668-4f83-921b-71e05844a82e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.404248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 476/964] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:13:02.408728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1e348-f74f-4051-b364-38794a9442ca None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.435650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.436368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.436645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.436911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.437564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.438921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:02.450795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:02.451853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:02.452739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 489/965] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:02.454537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.518927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:02.523071 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1e348-f74f-4051-b364-38794a9442ca None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.535049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:02.560304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.560972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.561260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.561513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.562220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:02.562526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.563527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075780c-9cbe-4949-8170-3a3388cecc3b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.563920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.574552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 477/966] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:02.574552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.604453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.605347 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.605696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.606041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.617618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.619729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:02.638490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:02.639767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:02.640877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:02.640877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:02.640877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:02.640877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:02.640877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 490/967] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:02.640877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.687490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ae7b-7ab7-4c6c-83bf-cd10d89c2ba4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.726515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.727189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.727506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.728722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.729556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:02.729874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.731636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.737398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 478/968] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:02.742709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.753932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56772f0-dc67-426e-95d3-c9a9e21ca6bb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.769499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:02.786686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:02.791866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae42fb-ef97-4c3e-9acc-5665d4ea122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.792535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae42fb-ef97-4c3e-9acc-5665d4ea122f None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.792822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae42fb-ef97-4c3e-9acc-5665d4ea122f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.793102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae42fb-ef97-4c3e-9acc-5665d4ea122f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.793866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae42fb-ef97-4c3e-9acc-5665d4ea122f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.795233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae42fb-ef97-4c3e-9acc-5665d4ea122f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.823365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 491/969] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:13:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:13:02.823365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae42fb-ef97-4c3e-9acc-5665d4ea122f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.865745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd6158-249c-45f6-a54a-475a97927fcb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.911497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.912269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.912618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.912957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.913873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:02.914246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.915909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:02.929482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 479/970] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:02.938063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:02.944109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae42fb-ef97-4c3e-9acc-5665d4ea122f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:02.990940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:02.991827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:02.992270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:02.992618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:02.994472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:02.994862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:02.996544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:03.009555 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 492/971] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:03.046794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:03.054811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829a1a0-b946-4506-b5c5-854cdbc1017c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:03.093618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:03.094449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:03.094734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:03.095005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:03.103963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:03.105529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:13:03.121819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:03.122814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:03.124264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 480/972] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:13:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:03.129416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:03.168381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:03.240338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc92f5a-c19c-4b9e-9031-0874487273b7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:03.279589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8fc26-b396-49bd-8d46-4f84eac5638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:03.280252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8fc26-b396-49bd-8d46-4f84eac5638b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:03.280530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8fc26-b396-49bd-8d46-4f84eac5638b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:03.280801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8fc26-b396-49bd-8d46-4f84eac5638b None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:03.281587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8fc26-b396-49bd-8d46-4f84eac5638b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:03.281892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8fc26-b396-49bd-8d46-4f84eac5638b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:03.283321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8fc26-b396-49bd-8d46-4f84eac5638b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:03.296973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 493/973] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:13:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:03.302422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:03.410697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cead-03f6-46d6-8633-6e59b603ff3d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:03.446695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824a798-c3b5-4947-9312-e74909b10927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:03.447673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a798-c3b5-4947-9312-e74909b10927 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:03.448002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a798-c3b5-4947-9312-e74909b10927 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:03.448287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a798-c3b5-4947-9312-e74909b10927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:03.449136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824a798-c3b5-4947-9312-e74909b10927 None admin] RBAC: Adding query filter params (role.id=883cfd8ec8cd40b583eea092a23386af, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=ba50db32ce9f444e919e667305594d4a) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:03.449451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824a798-c3b5-4947-9312-e74909b10927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:03.450885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824a798-c3b5-4947-9312-e74909b10927 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:03.457832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 481/974] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:03 2022] GET /identity/v3/role_assignments?user.id=ba50db32ce9f444e919e667305594d4a&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=883cfd8ec8cd40b583eea092a23386af => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:04.099246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617a73-e66f-4abe-a165-f5321f2034bd None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:04.099931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617a73-e66f-4abe-a165-f5321f2034bd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:04.099931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617a73-e66f-4abe-a165-f5321f2034bd None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:04.101483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 494/975] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:04.107585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e53208-3119-4845-a2ba-f9234680a51e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:04.107999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e53208-3119-4845-a2ba-f9234680a51e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:04.108187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e53208-3119-4845-a2ba-f9234680a51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:04.126072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d2e53208-3119-4845-a2ba-f9234680a51e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:04.127645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d2e53208-3119-4845-a2ba-f9234680a51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:04.142148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:04.200487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 482/976] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:04.225969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15617a73-e66f-4abe-a165-f5321f2034bd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:04.323774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:04.331493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15617a73-e66f-4abe-a165-f5321f2034bd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:04.370862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:04.371470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:04.371750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:04.372021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:04.372733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:04.374157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:04.375477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING keystone.server.flask.common [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 28 17:13:04.390199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.sql.core [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 17:13:04.390199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 28 17:13:04.390199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [parameters: {'user_id': 'ac40f1a9af344fbbad6a1ff8f35890bd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 28 17:13:04.390199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80262) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result = self._query(query) May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application conn.query(q) May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result.read() May 28 17:13:04.406841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 17:13:04.407792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application next(self.gen) May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application yield resource May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application next(self.gen) May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.session.rollback() May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.force_reraise() May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise self.value May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application session.commit() May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 17:13:04.408771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._prepare_impl() May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.session.flush() May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._flush(objects) May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application compat.raise_( May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application flush_context.execute() May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rec.execute(self) May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 17:13:04.409684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result = connection._execute_20( May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 17:13:04.410769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result = self._query(query) May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application conn.query(q) May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result.read() May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 17:13:04.412884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac40f1a9af344fbbad6a1ff8f35890bd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:04.414900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 28 17:13:04.417561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 28 17:13:04.427882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 28 17:13:04.427882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 28 17:13:04.427882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 17:13:04.427882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 17:13:04.427882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 28 17:13:04.427882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:04.427882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 495/977] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 17:13:04 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:04.428796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e53208-3119-4845-a2ba-f9234680a51e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:04.535544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e53208-3119-4845-a2ba-f9234680a51e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:04.570599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d7c76-b0db-4b28-bbb6-7519aed8a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:04.572121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d7c76-b0db-4b28-bbb6-7519aed8a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:04.572455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d7c76-b0db-4b28-bbb6-7519aed8a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:04.572734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d7c76-b0db-4b28-bbb6-7519aed8a8c7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:04.580426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d7c76-b0db-4b28-bbb6-7519aed8a8c7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:04.581927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d7c76-b0db-4b28-bbb6-7519aed8a8c7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b44d7c76-b0db-4b28-bbb6-7519aed8a8c7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:13:04.597111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:04.598076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 17:13:04.599042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:04.604986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 483/978] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:13:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:04.611525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:04.727360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc872-7023-4b73-ae1d-7264a1539637 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:04.761228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c67ad2-d4b7-4aa6-814a-c0b11648b29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:04.761906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67ad2-d4b7-4aa6-814a-c0b11648b29b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:04.762181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67ad2-d4b7-4aa6-814a-c0b11648b29b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:04.762449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67ad2-d4b7-4aa6-814a-c0b11648b29b None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:04.763258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c67ad2-d4b7-4aa6-814a-c0b11648b29b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:04.763592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c67ad2-d4b7-4aa6-814a-c0b11648b29b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:04.765221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c67ad2-d4b7-4aa6-814a-c0b11648b29b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:04.806188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 496/979] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 17:13:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:05.201894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8f185-dc99-4ca9-b6b9-bb4ee7a5cd56 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.202450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8f185-dc99-4ca9-b6b9-bb4ee7a5cd56 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.202450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8f185-dc99-4ca9-b6b9-bb4ee7a5cd56 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.204022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 484/980] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:05.209498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47763103-1412-4fac-aef2-9f8488e412c6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.209958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47763103-1412-4fac-aef2-9f8488e412c6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.210036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47763103-1412-4fac-aef2-9f8488e412c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.227354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-47763103-1412-4fac-aef2-9f8488e412c6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:05.229730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-47763103-1412-4fac-aef2-9f8488e412c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:05.240284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:05.302868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 497/981] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:05.325779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8f185-dc99-4ca9-b6b9-bb4ee7a5cd56 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:05.336613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:05.372771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc64e-2a9f-46ff-a0f9-7dc678bbc60c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.373107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc64e-2a9f-46ff-a0f9-7dc678bbc60c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.373351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc64e-2a9f-46ff-a0f9-7dc678bbc60c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.375192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 498/982] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:13:05.380043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a633a2-2efb-4dcb-b57a-53e1cdc0938b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.380333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a633a2-2efb-4dcb-b57a-53e1cdc0938b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.380793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a633a2-2efb-4dcb-b57a-53e1cdc0938b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.397440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b7a633a2-2efb-4dcb-b57a-53e1cdc0938b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:05.398635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b7a633a2-2efb-4dcb-b57a-53e1cdc0938b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:05.438402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8f185-dc99-4ca9-b6b9-bb4ee7a5cd56 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:05.472509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 499/983] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:05.477031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:05.477991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.478471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.478754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.479449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:05.480817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:05.494199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:05.495231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:05.496126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:05.496884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a633a2-2efb-4dcb-b57a-53e1cdc0938b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:05.497596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 485/984] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:13:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:05.501703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:05.578639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:05.579239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:05.613790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6336a-5fbf-4708-be65-e3f645195999 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:05.615224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a633a2-2efb-4dcb-b57a-53e1cdc0938b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:05.651013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:05.651711 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.651993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.652261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.652883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72620f-d26e-49fb-bbd2-f23bfa1081a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:05.653000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:05.653260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:05.653601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72620f-d26e-49fb-bbd2-f23bfa1081a5 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.653891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72620f-d26e-49fb-bbd2-f23bfa1081a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.654169 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72620f-d26e-49fb-bbd2-f23bfa1081a5 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.654616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:05.654956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72620f-d26e-49fb-bbd2-f23bfa1081a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:05.656440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72620f-d26e-49fb-bbd2-f23bfa1081a5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:05.664589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 486/985] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:13:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:05.665865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:05.676337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 500/986] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:13:05 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:13:05.782222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360c8e7-467e-48ac-9f6f-b666deee594d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.782403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360c8e7-467e-48ac-9f6f-b666deee594d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.782687 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360c8e7-467e-48ac-9f6f-b666deee594d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.784385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 501/987] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:13:05.788192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fe4fd-ec9a-4142-aadb-ffe9cacff3ed None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:05.790698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370048f-9866-41be-8b4c-c3fcb9fcf8c3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.791046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370048f-9866-41be-8b4c-c3fcb9fcf8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.791333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370048f-9866-41be-8b4c-c3fcb9fcf8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.808733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a370048f-9866-41be-8b4c-c3fcb9fcf8c3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:05.809967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a370048f-9866-41be-8b4c-c3fcb9fcf8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:05.830425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:05.836577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:05.837848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:05.838133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:05.838408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:05.839138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:05.839476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:05.840881 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:05.851091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 487/988] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:13:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:05.852299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:05.891252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 502/989] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:05.913609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370048f-9866-41be-8b4c-c3fcb9fcf8c3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:05.968036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e656-00cd-424f-b51b-6c86fb9f82fa None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:05.978846 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:06.004582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:06.005252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:06.005542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:06.005819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:06.006629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] RBAC: Adding query filter params (interface=admin, service_id=4c051b0911b94807a637a128f045d131, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:06.006946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:06.008344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:06.014200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 488/990] 10.222.0.33 () {56 vars in 1249 bytes} [Sat May 28 17:13:05 2022] GET /identity/v3/endpoints?service_id=4c051b0911b94807a637a128f045d131&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:06.020162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:06.022821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370048f-9866-41be-8b4c-c3fcb9fcf8c3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:06.060183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:06.061210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:06.061643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:06.062132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] PATH_INFO: `/v3/services/image` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:06.062955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:06.064614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:06.077790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:06.079022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:06.080128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:06.082059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:06.083269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 503/991] 10.222.0.33 () {56 vars in 1105 bytes} [Sat May 28 17:13:05 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:06.089745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:06.133553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3f2d7-80fb-4c07-ad55-e5c97a795ab4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:06.177706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887dc5f-c025-409b-89ee-01a2be3e35a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:06.178383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887dc5f-c025-409b-89ee-01a2be3e35a7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:06.178667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887dc5f-c025-409b-89ee-01a2be3e35a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:06.178943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887dc5f-c025-409b-89ee-01a2be3e35a7 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:06.179667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887dc5f-c025-409b-89ee-01a2be3e35a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:06.181475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887dc5f-c025-409b-89ee-01a2be3e35a7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:06.199409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 489/992] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:13:06 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:13:06.204322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa334d0-9b53-43b0-b2e3-004ad835cb19 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:06.235588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:06.246518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dd41a-e10b-4d40-8a1c-d209aba344e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:06.247203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dd41a-e10b-4d40-8a1c-d209aba344e4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:06.247525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dd41a-e10b-4d40-8a1c-d209aba344e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:06.247778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dd41a-e10b-4d40-8a1c-d209aba344e4 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:06.248508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dd41a-e10b-4d40-8a1c-d209aba344e4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:06.248813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dd41a-e10b-4d40-8a1c-d209aba344e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:06.250238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dd41a-e10b-4d40-8a1c-d209aba344e4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:06.261267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 504/993] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:06.262657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887dc5f-c025-409b-89ee-01a2be3e35a7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:06.397972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887dc5f-c025-409b-89ee-01a2be3e35a7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:06.421709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:06.433661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced4de8-a03f-469c-b2f2-52a86c6df55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:06.434333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced4de8-a03f-469c-b2f2-52a86c6df55a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:06.434612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced4de8-a03f-469c-b2f2-52a86c6df55a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:06.434885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced4de8-a03f-469c-b2f2-52a86c6df55a None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:06.435617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced4de8-a03f-469c-b2f2-52a86c6df55a None admin] RBAC: Adding query filter params (type=image) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:06.435925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced4de8-a03f-469c-b2f2-52a86c6df55a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:06.437290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced4de8-a03f-469c-b2f2-52a86c6df55a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:06.442024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 490/994] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:07.951824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ddab4-f9b5-405b-8dd3-38c2e33fb940 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:07.952354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ddab4-f9b5-405b-8dd3-38c2e33fb940 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:07.952474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ddab4-f9b5-405b-8dd3-38c2e33fb940 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:07.953928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 505/995] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:07.959236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4e9a8-c434-454f-9a33-a720a6c20008 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:07.959683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4e9a8-c434-454f-9a33-a720a6c20008 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:07.959864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4e9a8-c434-454f-9a33-a720a6c20008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:07.978164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2ca4e9a8-c434-454f-9a33-a720a6c20008 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:07.979576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2ca4e9a8-c434-454f-9a33-a720a6c20008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:08.049496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 491/996] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:08.072338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ddab4-f9b5-405b-8dd3-38c2e33fb940 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:08.175752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ddab4-f9b5-405b-8dd3-38c2e33fb940 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:08.210179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:08.210977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.211259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.211561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.212624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:08.213657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:08.225957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:08.227035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:08.229822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:08.231751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 506/997] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:13:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:08.234144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4e9a8-c434-454f-9a33-a720a6c20008 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:08.348359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4e9a8-c434-454f-9a33-a720a6c20008 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:08.366609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:08.386555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09a748-b26d-49f9-a1be-d9f381f655da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:08.387231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09a748-b26d-49f9-a1be-d9f381f655da None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.387544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09a748-b26d-49f9-a1be-d9f381f655da None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.387825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09a748-b26d-49f9-a1be-d9f381f655da None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.388540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09a748-b26d-49f9-a1be-d9f381f655da None admin] RBAC: Adding query filter params (name=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:08.388853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09a748-b26d-49f9-a1be-d9f381f655da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:08.390227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09a748-b26d-49f9-a1be-d9f381f655da None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:08.400080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 492/998] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:13:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:08.401658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:08.438972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79ad4e-c892-4e30-8213-7a665edfeaa2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.439262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79ad4e-c892-4e30-8213-7a665edfeaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.439577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79ad4e-c892-4e30-8213-7a665edfeaa2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.441069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 493/999] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:13:08.445857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd952a11-a243-4337-8444-587c28f720fe None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.446136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd952a11-a243-4337-8444-587c28f720fe None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.446506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd952a11-a243-4337-8444-587c28f720fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.463084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:08.466192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bd952a11-a243-4337-8444-587c28f720fe None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:08.467470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bd952a11-a243-4337-8444-587c28f720fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:08.515537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49da076-03a9-483e-86f3-648d78df171a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:08.543232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 494/1000] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:08.554384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:08.555056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.555372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.556147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.556896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:08.557265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:08.558670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:08.565694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 507/1001] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:13:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:08.568378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd952a11-a243-4337-8444-587c28f720fe None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:08.569625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:08.674577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd952a11-a243-4337-8444-587c28f720fe None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:08.676041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6cd18-8948-49ac-9c7d-70fdea239764 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:08.688599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:08.695919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:08.711351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b8183-ef40-4f31-8af5-c8ea137473ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:08.712303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b8183-ef40-4f31-8af5-c8ea137473ce None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.712590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d243714-ad63-4ae3-b97e-0af10f463bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:08.712719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b8183-ef40-4f31-8af5-c8ea137473ce None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.713034 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b8183-ef40-4f31-8af5-c8ea137473ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.713129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d243714-ad63-4ae3-b97e-0af10f463bda None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.713407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d243714-ad63-4ae3-b97e-0af10f463bda None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.713680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d243714-ad63-4ae3-b97e-0af10f463bda None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.713866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b8183-ef40-4f31-8af5-c8ea137473ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:08.714469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d243714-ad63-4ae3-b97e-0af10f463bda None admin] RBAC: Adding query filter params (interface=public, service_id=04320854410749c39c80189e6acfc9f6, region_id=RegionOne) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:08.714781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d243714-ad63-4ae3-b97e-0af10f463bda None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:08.715480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b8183-ef40-4f31-8af5-c8ea137473ce None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:08.716185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d243714-ad63-4ae3-b97e-0af10f463bda None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:08.722329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 508/1002] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 17:13:08 2022] GET /identity/v3/endpoints?service_id=04320854410749c39c80189e6acfc9f6&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:08.723446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d4d6-5e47-423f-8357-db251b0a7d72 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.723735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d4d6-5e47-423f-8357-db251b0a7d72 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.724011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d4d6-5e47-423f-8357-db251b0a7d72 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.727381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 509/1003] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:08.728054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7d4d6-5e47-423f-8357-db251b0a7d72 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-731b8183-ef40-4f31-8af5-c8ea137473ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:08.728672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:08.729628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 17:13:08.730544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:08.733739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 495/1004] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 17:13:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:08.733739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5b70-00bf-4bcf-95a1-8617c5698d75 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.733739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5b70-00bf-4bcf-95a1-8617c5698d75 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.733999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5b70-00bf-4bcf-95a1-8617c5698d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.751625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b0ac5b70-00bf-4bcf-95a1-8617c5698d75 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:08.752804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b0ac5b70-00bf-4bcf-95a1-8617c5698d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:08.817203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 496/1005] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:08.818215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac5b70-00bf-4bcf-95a1-8617c5698d75 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:08.834957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7d4d6-5e47-423f-8357-db251b0a7d72 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:08.870652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572220a5-5d2f-446f-ad31-cbe76659bac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:08.871358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572220a5-5d2f-446f-ad31-cbe76659bac3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.871661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572220a5-5d2f-446f-ad31-cbe76659bac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.871936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572220a5-5d2f-446f-ad31-cbe76659bac3 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.872619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572220a5-5d2f-446f-ad31-cbe76659bac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:08.873983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572220a5-5d2f-446f-ad31-cbe76659bac3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:08.880082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 510/1006] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:13:08 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:13:08.880826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572220a5-5d2f-446f-ad31-cbe76659bac3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:08.929204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac5b70-00bf-4bcf-95a1-8617c5698d75 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:08.943388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:08.949822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:08.975020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:08.975731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:08.976725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:08.976725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:08.977008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:08.977315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:08.979521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:08.985580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 497/1007] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:13:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:08.991196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:09.013001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572220a5-5d2f-446f-ad31-cbe76659bac3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:09.052104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c90d8-fbfd-4a0e-b677-dfa8104bd71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:09.053093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c90d8-fbfd-4a0e-b677-dfa8104bd71e None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:09.053552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c90d8-fbfd-4a0e-b677-dfa8104bd71e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:09.053839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c90d8-fbfd-4a0e-b677-dfa8104bd71e None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:09.054597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c90d8-fbfd-4a0e-b677-dfa8104bd71e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:09.056806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c90d8-fbfd-4a0e-b677-dfa8104bd71e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:09.074276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 511/1008] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:13:08 2022] POST /identity/v3/services => generated 242 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:13:09.111653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eb36b-5afa-4178-9e40-ce45e94ef9e2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:09.157271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88ae84-469a-4ca0-993e-4b3c4c21cb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:09.157965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88ae84-469a-4ca0-993e-4b3c4c21cb06 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:09.158251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88ae84-469a-4ca0-993e-4b3c4c21cb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:09.158532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88ae84-469a-4ca0-993e-4b3c4c21cb06 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:09.159252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88ae84-469a-4ca0-993e-4b3c4c21cb06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:09.159890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88ae84-469a-4ca0-993e-4b3c4c21cb06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:09.161187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88ae84-469a-4ca0-993e-4b3c4c21cb06 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:09.172760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 498/1009] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 17:13:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:09.174118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c90d8-fbfd-4a0e-b677-dfa8104bd71e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:09.286095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c90d8-fbfd-4a0e-b677-dfa8104bd71e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:09.296379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:09.323571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cabcb8-21e2-46ed-ba76-069c43bd0734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:09.324246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cabcb8-21e2-46ed-ba76-069c43bd0734 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:09.324581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cabcb8-21e2-46ed-ba76-069c43bd0734 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:09.324802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cabcb8-21e2-46ed-ba76-069c43bd0734 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:09.325541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cabcb8-21e2-46ed-ba76-069c43bd0734 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:09.326925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cabcb8-21e2-46ed-ba76-069c43bd0734 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:09.343227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 512/1010] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:13:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:13:11.140263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a203-20e7-4f34-89e3-dc017c5873d2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.140766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a203-20e7-4f34-89e3-dc017c5873d2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.140766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a203-20e7-4f34-89e3-dc017c5873d2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.143382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 499/1011] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:11.147477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4de81-1c40-4806-b16b-67f55f8c13fc None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.147893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4de81-1c40-4806-b16b-67f55f8c13fc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.148065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4de81-1c40-4806-b16b-67f55f8c13fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.165802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c8b4de81-1c40-4806-b16b-67f55f8c13fc None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:11.167006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c8b4de81-1c40-4806-b16b-67f55f8c13fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:11.241584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 513/1012] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:11 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:11.264348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370a203-20e7-4f34-89e3-dc017c5873d2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:11.280663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63be600-ac39-4795-a0c9-6ab37d930f88 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.280952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63be600-ac39-4795-a0c9-6ab37d930f88 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.281227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63be600-ac39-4795-a0c9-6ab37d930f88 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.283378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 514/1013] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:13:11.287738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0418-b514-4c14-b048-4e7be0549ffc None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.288019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0418-b514-4c14-b048-4e7be0549ffc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.288293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0418-b514-4c14-b048-4e7be0549ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.299426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:11.307369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2bda0418-b514-4c14-b048-4e7be0549ffc None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:11.308569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2bda0418-b514-4c14-b048-4e7be0549ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:11.374474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370a203-20e7-4f34-89e3-dc017c5873d2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:11.383764 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 515/1014] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:11 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:11.403047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda0418-b514-4c14-b048-4e7be0549ffc None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:11.418001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:11.419010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.419294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.420053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.421027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:11.422607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:11.430380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:11.436165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:11.437662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 17:13:11.438530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:11.442651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 500/1015] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:13:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:11.443532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:11.512591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda0418-b514-4c14-b048-4e7be0549ffc None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:11.547622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:11.549377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:11.550217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.550606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.550845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] PATH_INFO: `/v3/services/image` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.551526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:11.552912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:11.556531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348e888-6ebc-476b-8fa1-16fcb10bea93 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:11.565125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:11.566133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:11.567004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 516/1016] 10.222.0.33 () {56 vars in 1105 bytes} [Sat May 28 17:13:11 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:11.572301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:11.594335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbb512-a9c9-4d95-bd76-653687ad9cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:11.595006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbb512-a9c9-4d95-bd76-653687ad9cda None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.595285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbb512-a9c9-4d95-bd76-653687ad9cda None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.595590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbb512-a9c9-4d95-bd76-653687ad9cda None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.596296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbb512-a9c9-4d95-bd76-653687ad9cda None admin] RBAC: Adding query filter params (name=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:11.596612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbb512-a9c9-4d95-bd76-653687ad9cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:11.598057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbb512-a9c9-4d95-bd76-653687ad9cda None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:11.608719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 501/1017] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:13:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:11.608719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec8ca-958d-4a0d-9ab8-6d62fd6d48ac None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.608719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec8ca-958d-4a0d-9ab8-6d62fd6d48ac None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.608719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec8ca-958d-4a0d-9ab8-6d62fd6d48ac None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.608719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 502/1018] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:11.609558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fec8ca-958d-4a0d-9ab8-6d62fd6d48ac None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:11.669884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:11.685349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc119133-426f-4ca4-8703-f2134999210d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:11.709213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fec8ca-958d-4a0d-9ab8-6d62fd6d48ac None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:11.724064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6aa41-363a-46de-a5b5-b8a6382bd71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:11.724735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6aa41-363a-46de-a5b5-b8a6382bd71e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.725023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6aa41-363a-46de-a5b5-b8a6382bd71e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.725297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6aa41-363a-46de-a5b5-b8a6382bd71e None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.726009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6aa41-363a-46de-a5b5-b8a6382bd71e None admin] RBAC: Adding query filter params (name=image) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:11.726319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6aa41-363a-46de-a5b5-b8a6382bd71e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:11.727816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6aa41-363a-46de-a5b5-b8a6382bd71e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:11.732751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 517/1019] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:11.733926 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68f7b-904e-4691-8f25-939c9c269cda None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.734218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68f7b-904e-4691-8f25-939c9c269cda None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.734498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68f7b-904e-4691-8f25-939c9c269cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.745323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:11.745981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.746267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.746539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.747238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:11.747572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:11.748920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:11.750875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d7a68f7b-904e-4691-8f25-939c9c269cda None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:11.752101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d7a68f7b-904e-4691-8f25-939c9c269cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:11.753768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 503/1020] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 17:13:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:11.754545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:11.778022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:11.823466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 518/1021] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:11 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:11.824285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a68f7b-904e-4691-8f25-939c9c269cda None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:11.858586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84dacf-f145-46c4-8175-3f871f02dd01 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:11.892886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:11.893551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.893832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.894110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.894816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] RBAC: Adding query filter params (type=image) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:11.895123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:11.896523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:11.901451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 504/1022] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:11.902220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:11.909533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:11.930114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a68f7b-904e-4691-8f25-939c9c269cda None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:11.964124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97245837-3e8f-4fb5-9c0d-3c32eab5f087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:11.964775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97245837-3e8f-4fb5-9c0d-3c32eab5f087 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:11.965054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97245837-3e8f-4fb5-9c0d-3c32eab5f087 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:11.965325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97245837-3e8f-4fb5-9c0d-3c32eab5f087 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:11.966065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97245837-3e8f-4fb5-9c0d-3c32eab5f087 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:11.967466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97245837-3e8f-4fb5-9c0d-3c32eab5f087 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:11.986859 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 519/1023] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:13:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:13:11.987932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97245837-3e8f-4fb5-9c0d-3c32eab5f087 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:12.000847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:12.003900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94142fb8-2033-4c84-bfc1-c4d156889c4d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:12.051740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247da7c4-81d5-4740-a1b3-9e6baed64d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:12.052571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247da7c4-81d5-4740-a1b3-9e6baed64d6c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:12.052853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247da7c4-81d5-4740-a1b3-9e6baed64d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:12.053127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247da7c4-81d5-4740-a1b3-9e6baed64d6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:12.054520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247da7c4-81d5-4740-a1b3-9e6baed64d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:12.056613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247da7c4-81d5-4740-a1b3-9e6baed64d6c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:12.058779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 505/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:12.064655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247da7c4-81d5-4740-a1b3-9e6baed64d6c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:12.114327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97245837-3e8f-4fb5-9c0d-3c32eab5f087 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:12.152306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:12.153087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:12.153435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:12.153715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:12.154520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] RBAC: Adding query filter params (interface=public, service_id=7b158a018bd74aa08fe4f1ffa4b4030a, region_id=RegionOne) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:12.154838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:12.156265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:12.164733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 520/1025] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 17:13:11 2022] GET /identity/v3/endpoints?service_id=7b158a018bd74aa08fe4f1ffa4b4030a&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:12.168019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:12.170889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:12.184263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247da7c4-81d5-4740-a1b3-9e6baed64d6c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:12.224722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448f376-2867-42a3-91c4-5583edb93d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:12.225405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448f376-2867-42a3-91c4-5583edb93d88 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:12.225698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448f376-2867-42a3-91c4-5583edb93d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:12.225972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448f376-2867-42a3-91c4-5583edb93d88 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:12.226698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448f376-2867-42a3-91c4-5583edb93d88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:12.228371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448f376-2867-42a3-91c4-5583edb93d88 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:12.259268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 506/1026] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:13:12 2022] POST /identity/v3/projects => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:13:12.295011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f37e8-aef1-45fa-8a9e-020340a075b3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:12.340647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d7f2b-3fdc-469d-a51f-cb0b9002cf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:12.341368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d7f2b-3fdc-469d-a51f-cb0b9002cf1f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:12.341647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d7f2b-3fdc-469d-a51f-cb0b9002cf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:12.341914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d7f2b-3fdc-469d-a51f-cb0b9002cf1f None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:12.342622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d7f2b-3fdc-469d-a51f-cb0b9002cf1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:12.345687 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d7f2b-3fdc-469d-a51f-cb0b9002cf1f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:12.356992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 521/1027] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:13:12 2022] GET /identity/v3/services => generated 1957 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:13:14.324158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2ce5f-00b1-42b9-9138-cdb01dcca9f5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.324684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2ce5f-00b1-42b9-9138-cdb01dcca9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.324684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2ce5f-00b1-42b9-9138-cdb01dcca9f5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.326148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 507/1028] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:14.331529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61d8e-8aa2-41ec-ac58-70b26aa22578 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.331982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61d8e-8aa2-41ec-ac58-70b26aa22578 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.332163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61d8e-8aa2-41ec-ac58-70b26aa22578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.350316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-95f61d8e-8aa2-41ec-ac58-70b26aa22578 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:14.351583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-95f61d8e-8aa2-41ec-ac58-70b26aa22578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:14.431248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:14.440516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 522/1029] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:14.465298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2ce5f-00b1-42b9-9138-cdb01dcca9f5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:14.469535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:14.542290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590267c8-2b0b-4b79-9088-3050f0084819 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.542620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590267c8-2b0b-4b79-9088-3050f0084819 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.542847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590267c8-2b0b-4b79-9088-3050f0084819 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.544420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 523/1030] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:14.549366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d91a84-758b-4507-99e7-417789b96e2e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.549653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d91a84-758b-4507-99e7-417789b96e2e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.549928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d91a84-758b-4507-99e7-417789b96e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.566886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-52d91a84-758b-4507-99e7-417789b96e2e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:14.568091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-52d91a84-758b-4507-99e7-417789b96e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:14.581013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2ce5f-00b1-42b9-9138-cdb01dcca9f5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:14.617716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79305a82-ca08-452d-9025-61e2eaf0d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:14.619124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305a82-ca08-452d-9025-61e2eaf0d235 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.619433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305a82-ca08-452d-9025-61e2eaf0d235 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.619711 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305a82-ca08-452d-9025-61e2eaf0d235 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.630741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79305a82-ca08-452d-9025-61e2eaf0d235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:14.639521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 524/1031] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-79305a82-ca08-452d-9025-61e2eaf0d235 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:14.643593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:14.644646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:14.645537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 508/1032] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 17:13:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:14.651031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d91a84-758b-4507-99e7-417789b96e2e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:14.667360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79305a82-ca08-452d-9025-61e2eaf0d235 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:14.737624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:14.760243 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d91a84-758b-4507-99e7-417789b96e2e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:14.779363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79305a82-ca08-452d-9025-61e2eaf0d235 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:14.787502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:14.797005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a9bc3-32ee-4344-b602-e3c5b7575dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:14.797674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9bc3-32ee-4344-b602-e3c5b7575dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.797949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9bc3-32ee-4344-b602-e3c5b7575dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.798222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9bc3-32ee-4344-b602-e3c5b7575dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.798977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a9bc3-32ee-4344-b602-e3c5b7575dc4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:14.799284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a9bc3-32ee-4344-b602-e3c5b7575dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:14.800676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a9bc3-32ee-4344-b602-e3c5b7575dc4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:14.809197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 525/1033] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 17:13:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:14.812739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbaf1b-c473-4e01-bebe-c116d8a26a2f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.813027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbaf1b-c473-4e01-bebe-c116d8a26a2f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.813310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbaf1b-c473-4e01-bebe-c116d8a26a2f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.813870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a75d5-cf59-4f6f-80b3-212091116dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:14.814486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a75d5-cf59-4f6f-80b3-212091116dad None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.814585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 526/1034] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:14.814774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a75d5-cf59-4f6f-80b3-212091116dad None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.815049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a75d5-cf59-4f6f-80b3-212091116dad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.815248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbaf1b-c473-4e01-bebe-c116d8a26a2f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:14.816319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a75d5-cf59-4f6f-80b3-212091116dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:14.817747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a75d5-cf59-4f6f-80b3-212091116dad None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:14.824692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 509/1035] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:14.824692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda6f3-c03c-41c3-82a1-b74441140507 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.824692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda6f3-c03c-41c3-82a1-b74441140507 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.824692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda6f3-c03c-41c3-82a1-b74441140507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.840703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e5cda6f3-c03c-41c3-82a1-b74441140507 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:14.841796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e5cda6f3-c03c-41c3-82a1-b74441140507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:14.908679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 510/1036] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:14.909479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cda6f3-c03c-41c3-82a1-b74441140507 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:14.923408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbaf1b-c473-4e01-bebe-c116d8a26a2f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:14.957179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:14.958070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:14.958351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:14.958611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:14.968077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:14.969548 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:14.971015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:14.988842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:14.989972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:14.991078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:14.991078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:14.991078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:14.991078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:14.991078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 527/1037] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:14.991078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:15.012526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cda6f3-c03c-41c3-82a1-b74441140507 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.033126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:15.046210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0affe75-272d-421d-9ac3-723fa4701b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.046885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0affe75-272d-421d-9ac3-723fa4701b34 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.047169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0affe75-272d-421d-9ac3-723fa4701b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.047483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0affe75-272d-421d-9ac3-723fa4701b34 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.048191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0affe75-272d-421d-9ac3-723fa4701b34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.049608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0affe75-272d-421d-9ac3-723fa4701b34 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.069812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 511/1038] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:13:14 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:13:15.070616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0affe75-272d-421d-9ac3-723fa4701b34 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:15.096399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06390ac-58e9-4e60-8449-f222621a57ea None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.135585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.136402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.136675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.136951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] PATH_INFO: `/v3/services/image` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.137622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.139059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:15.151636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:15.152762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:15.153790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 528/1039] 10.222.0.33 () {56 vars in 1105 bytes} [Sat May 28 17:13:14 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:15.158999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:15.202376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0affe75-272d-421d-9ac3-723fa4701b34 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.221005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:15.245228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.246080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.246368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.246649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.247422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:15.247737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.249114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.262971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 512/1040] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:15.268009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60cdf-16d4-40f8-8d85-a4b897df6a0d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.268587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:15.270401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:15.312335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.313006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.313282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.314172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.314291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] RBAC: Adding query filter params (name=image) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:15.314560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.318129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.324870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 529/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:15.330643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:15.380677 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63fb03-3382-4a4a-854e-03475257b7d5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.417337 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.418136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.418416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.418690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.426444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.427930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.442416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420acb7e-f325-4cf1-8111-9ecd89544a5b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:13:15.443090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:15.444137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:15.445175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 513/1042] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:13:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:15.450362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:15.483635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.484303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.484582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.485281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.485886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:15.486122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.487516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.490100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:15.497984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 530/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:15.498979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:15.533805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:15.563972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def161e1-f520-4d80-b8a3-9e07c39c1beb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.604281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.604960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.605244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.605519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.606370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:15.606687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.608138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.613016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab027ec-4e10-48ac-8e74-2e3789ab84d5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.631604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 514/1044] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:13:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:15.632424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:15.650919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882dc00-525f-40cb-945e-23f3a253e9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.651866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882dc00-525f-40cb-945e-23f3a253e9be None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.652344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882dc00-525f-40cb-945e-23f3a253e9be None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.652791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882dc00-525f-40cb-945e-23f3a253e9be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.653726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882dc00-525f-40cb-945e-23f3a253e9be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.655330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882dc00-525f-40cb-945e-23f3a253e9be None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.683374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 531/1045] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:13:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:13:15.735699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:15.742930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fa830-78a4-48e8-86cf-86227c9bd3dc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:15.791648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780318f8-fd91-45bd-925c-0ae328220ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:15.792329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780318f8-fd91-45bd-925c-0ae328220ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:15.792667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780318f8-fd91-45bd-925c-0ae328220ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:15.792951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780318f8-fd91-45bd-925c-0ae328220ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:15.877324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780318f8-fd91-45bd-925c-0ae328220ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:15.878127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780318f8-fd91-45bd-925c-0ae328220ae5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:15.968427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:15.980140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 515/1046] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:13:15.986366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882dc00-525f-40cb-945e-23f3a253e9be None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:16.084977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:16.099611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882dc00-525f-40cb-945e-23f3a253e9be None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:16.133859 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:16.134650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:16.134929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:16.135204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:16.145851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:16.147425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:16.164886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:16.166065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:16.166988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:16.166988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:16.166988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:16.166988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:16.167867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 532/1047] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:16.172052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780318f8-fd91-45bd-925c-0ae328220ae5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:16.278079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780318f8-fd91-45bd-925c-0ae328220ae5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:16.313166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:16.313838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:16.314124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:16.314404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:16.315129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:16.315487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:16.316835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:16.328344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 516/1048] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:16.334002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:16.440020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb1e4c-ad32-4136-a97f-48c2e27249c1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:16.474417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:16.475223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:16.475534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:16.475811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:16.483272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:16.484756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:16.487585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:13:16.501263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:16.502180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:16.503092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:13:16.503092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:13:16.503092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:13:16.503092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:13:16.503092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:13:16.503092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:13:16.503092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:16.503092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:16.507277 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 533/1049] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:13:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:16.508276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:16.559096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:16.608858 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56944ecb-d32a-44d8-b307-7516ad1411cd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:16.641722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aca8bd-8761-49d7-a45d-3580cd1885dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:16.642387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aca8bd-8761-49d7-a45d-3580cd1885dc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:16.642672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aca8bd-8761-49d7-a45d-3580cd1885dc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:16.642951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aca8bd-8761-49d7-a45d-3580cd1885dc None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:16.643766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aca8bd-8761-49d7-a45d-3580cd1885dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:16.644079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aca8bd-8761-49d7-a45d-3580cd1885dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:16.645506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aca8bd-8761-49d7-a45d-3580cd1885dc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:16.658853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 517/1050] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:13:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:16.664676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:16.778994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78647520-ac9b-4630-b5ab-a678db3787a5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:16.814881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acbbce-6293-45b9-9258-5f647a5ceb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:16.815480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acbbce-6293-45b9-9258-5f647a5ceb08 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:16.815762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acbbce-6293-45b9-9258-5f647a5ceb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:16.816031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acbbce-6293-45b9-9258-5f647a5ceb08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:16.816858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acbbce-6293-45b9-9258-5f647a5ceb08 None admin] RBAC: Adding query filter params (role.id=883cfd8ec8cd40b583eea092a23386af, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=5b9888d2731f4117b5fc2c3a720931c4) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:16.817166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acbbce-6293-45b9-9258-5f647a5ceb08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:16.818631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acbbce-6293-45b9-9258-5f647a5ceb08 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:16.831490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 534/1051] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:16 2022] GET /identity/v3/role_assignments?user.id=5b9888d2731f4117b5fc2c3a720931c4&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=883cfd8ec8cd40b583eea092a23386af => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:17.356637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdf04a-23e9-4fd9-8015-703ce37d4145 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:17.356974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdf04a-23e9-4fd9-8015-703ce37d4145 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:17.357198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdf04a-23e9-4fd9-8015-703ce37d4145 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:17.358718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 518/1052] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:17.364164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc17a10-9d45-410b-b6ae-3f182379f942 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:17.364460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc17a10-9d45-410b-b6ae-3f182379f942 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:17.364737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc17a10-9d45-410b-b6ae-3f182379f942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:17.383836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2dc17a10-9d45-410b-b6ae-3f182379f942 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:17.385172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2dc17a10-9d45-410b-b6ae-3f182379f942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:17.444931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:17.455023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 535/1053] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:17.479857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdf04a-23e9-4fd9-8015-703ce37d4145 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:17.583249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdf04a-23e9-4fd9-8015-703ce37d4145 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:17.594582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:17.621239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6eec67-2d8a-47e1-9938-e050cd37ceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:17.622277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6eec67-2d8a-47e1-9938-e050cd37ceec None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:17.622702 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6eec67-2d8a-47e1-9938-e050cd37ceec None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:17.623119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6eec67-2d8a-47e1-9938-e050cd37ceec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:17.634888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6eec67-2d8a-47e1-9938-e050cd37ceec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3c6eec67-2d8a-47e1-9938-e050cd37ceec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:17.642791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:17.643895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:17.644746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:17.647488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 519/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:13:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:17.650948 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc17a10-9d45-410b-b6ae-3f182379f942 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:17.764670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc17a10-9d45-410b-b6ae-3f182379f942 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:17.800790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33d7dc-4bd0-4eb3-bb64-4dfac986b88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:17.801460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33d7dc-4bd0-4eb3-bb64-4dfac986b88c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:17.801742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33d7dc-4bd0-4eb3-bb64-4dfac986b88c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:17.802015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33d7dc-4bd0-4eb3-bb64-4dfac986b88c None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:17.802768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33d7dc-4bd0-4eb3-bb64-4dfac986b88c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:17.803079 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33d7dc-4bd0-4eb3-bb64-4dfac986b88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:17.804506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33d7dc-4bd0-4eb3-bb64-4dfac986b88c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:17.817215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 536/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:13:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:17.818474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6eec67-2d8a-47e1-9938-e050cd37ceec None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:17.921581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6eec67-2d8a-47e1-9938-e050cd37ceec None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:17.942074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762568-d083-4f36-a7b9-999e00bbf2a4 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:17.942378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762568-d083-4f36-a7b9-999e00bbf2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:17.942639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762568-d083-4f36-a7b9-999e00bbf2a4 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:17.944175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 537/1056] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:13:17.949429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed240fca-80fa-4380-9e9d-374b1ceedc4b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:17.949722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed240fca-80fa-4380-9e9d-374b1ceedc4b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:17.950001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed240fca-80fa-4380-9e9d-374b1ceedc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:17.956538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878574b5-05c8-4021-a78c-4d5d65ccf208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:17.957413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878574b5-05c8-4021-a78c-4d5d65ccf208 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:17.957697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878574b5-05c8-4021-a78c-4d5d65ccf208 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:17.957977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878574b5-05c8-4021-a78c-4d5d65ccf208 None admin] PATH_INFO: `/v3/users/9a19c784d2094bfe99cb13fa37ad5e7b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:17.968781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ed240fca-80fa-4380-9e9d-374b1ceedc4b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:17.969991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ed240fca-80fa-4380-9e9d-374b1ceedc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:17.973529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878574b5-05c8-4021-a78c-4d5d65ccf208 None admin] RBAC: Authorizing `identity:get_user(user_id=9a19c784d2094bfe99cb13fa37ad5e7b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:17.974967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878574b5-05c8-4021-a78c-4d5d65ccf208 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:17.981946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 520/1057] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:17 2022] GET /identity/v3/users/9a19c784d2094bfe99cb13fa37ad5e7b => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:17.982904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878574b5-05c8-4021-a78c-4d5d65ccf208 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:18.013947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:18.026505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:18.045889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 538/1058] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:18.067568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed240fca-80fa-4380-9e9d-374b1ceedc4b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:18.092435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878574b5-05c8-4021-a78c-4d5d65ccf208 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:18.132351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804b783-a2c2-4806-834f-0506a01c1d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:18.133001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804b783-a2c2-4806-834f-0506a01c1d1a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:18.133284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804b783-a2c2-4806-834f-0506a01c1d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:18.133562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804b783-a2c2-4806-834f-0506a01c1d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:18.181326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed240fca-80fa-4380-9e9d-374b1ceedc4b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:18.206499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804b783-a2c2-4806-834f-0506a01c1d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:18.207986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804b783-a2c2-4806-834f-0506a01c1d1a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:18.219661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911e48b-2191-4fa5-9136-6ee3f4d8c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:18.220311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e48b-2191-4fa5-9136-6ee3f4d8c935 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:18.220594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e48b-2191-4fa5-9136-6ee3f4d8c935 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:18.220874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e48b-2191-4fa5-9136-6ee3f4d8c935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:18.235730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:18.280908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:18.296219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911e48b-2191-4fa5-9136-6ee3f4d8c935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:18.297673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911e48b-2191-4fa5-9136-6ee3f4d8c935 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:18.313586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 521/1059] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:13:18.314860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804b783-a2c2-4806-834f-0506a01c1d1a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:18.404929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 539/1060] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 17:13:18.411210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911e48b-2191-4fa5-9136-6ee3f4d8c935 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:18.422238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804b783-a2c2-4806-834f-0506a01c1d1a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:18.451932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:18.460997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5477e34-6bc4-459b-b85d-205caefa6c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:18.461901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5477e34-6bc4-459b-b85d-205caefa6c22 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:18.462340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5477e34-6bc4-459b-b85d-205caefa6c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:18.462610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5477e34-6bc4-459b-b85d-205caefa6c22 None admin] PATH_INFO: `/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:18.463838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5477e34-6bc4-459b-b85d-205caefa6c22 None admin] RBAC: Authorizing `identity:get_project(project_id=4a472dd1d1dd4620a18afe26cb3afd58)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:18.465257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5477e34-6bc4-459b-b85d-205caefa6c22 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:18.467119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 522/1061] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:18 2022] GET /identity/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:18.472764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5477e34-6bc4-459b-b85d-205caefa6c22 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:18.503295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:18.518838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911e48b-2191-4fa5-9136-6ee3f4d8c935 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:18.555418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:18.556324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:18.556580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:18.556849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:18.567367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:18.568830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:18.582280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5477e34-6bc4-459b-b85d-205caefa6c22 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:18.587899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:18.588877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:18.589839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:18.589839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:18.589839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:18.589839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:18.589839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 540/1062] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:18.595070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:18.620592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092368e-26d1-433f-9ba1-4fd1fcdd0968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:18.621582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092368e-26d1-433f-9ba1-4fd1fcdd0968 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:18.621582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092368e-26d1-433f-9ba1-4fd1fcdd0968 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:18.621808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092368e-26d1-433f-9ba1-4fd1fcdd0968 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:18.622609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092368e-26d1-433f-9ba1-4fd1fcdd0968 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=4a472dd1d1dd4620a18afe26cb3afd58, user.id=9a19c784d2094bfe99cb13fa37ad5e7b) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:18.622927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092368e-26d1-433f-9ba1-4fd1fcdd0968 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:18.624352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092368e-26d1-433f-9ba1-4fd1fcdd0968 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:18.639368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 523/1063] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:18 2022] GET /identity/v3/role_assignments?user.id=9a19c784d2094bfe99cb13fa37ad5e7b&scope.project.id=4a472dd1d1dd4620a18afe26cb3afd58&role.id=e07cc5009f624eb9912a837c934e68aa => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:18.709672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02a26f-d5db-488d-8cca-fa151cd8409b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:18.757298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f82ea8-6687-41ee-aec1-f40e949cb328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:18.757975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f82ea8-6687-41ee-aec1-f40e949cb328 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:18.758254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f82ea8-6687-41ee-aec1-f40e949cb328 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:18.758527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f82ea8-6687-41ee-aec1-f40e949cb328 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:18.760155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f82ea8-6687-41ee-aec1-f40e949cb328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:18.760454 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f82ea8-6687-41ee-aec1-f40e949cb328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:18.761870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f82ea8-6687-41ee-aec1-f40e949cb328 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:18.769355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:18.783377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 541/1064] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:19.048570 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f4470-e47b-4113-b10d-f65c94206557 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:19.048764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f4470-e47b-4113-b10d-f65c94206557 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:19.049044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f4470-e47b-4113-b10d-f65c94206557 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:19.050589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 524/1065] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:19.056249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29102c-c302-4171-a60a-f276a5d1c332 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:19.056540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29102c-c302-4171-a60a-f276a5d1c332 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:19.056819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29102c-c302-4171-a60a-f276a5d1c332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:19.073950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8e29102c-c302-4171-a60a-f276a5d1c332 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:19.075147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8e29102c-c302-4171-a60a-f276a5d1c332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:19.142315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 542/1066] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:19.164838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f4470-e47b-4113-b10d-f65c94206557 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:19.226013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:19.268159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f4470-e47b-4113-b10d-f65c94206557 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:19.301591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b6366-e618-4024-9cde-929179262b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:19.302324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b6366-e618-4024-9cde-929179262b5d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:19.302606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b6366-e618-4024-9cde-929179262b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:19.302880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b6366-e618-4024-9cde-929179262b5d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:19.312878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b6366-e618-4024-9cde-929179262b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-487b6366-e618-4024-9cde-929179262b5d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:19.321608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:19.322576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:19.323584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 525/1067] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 17:13:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:19.328982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29102c-c302-4171-a60a-f276a5d1c332 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:19.427507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29102c-c302-4171-a60a-f276a5d1c332 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:19.454203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:19.463365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:19.464201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:19.464487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:19.464765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:19.465518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:19.465830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:19.467217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:19.475295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 543/1068] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 17:13:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:19.481416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b6366-e618-4024-9cde-929179262b5d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:19.579541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b6366-e618-4024-9cde-929179262b5d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:19.598950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:19.613422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:19.614208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:19.614491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:19.614771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:19.625614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:19.627056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:19.643839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:19.644903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:19.645849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:19.645849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:19.645849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:19.645849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:19.645849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 526/1069] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:19.651593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:19.753985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec0e24-86a3-49a3-9d57-122993520dce None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:19.787213 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:19.787900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:19.788183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:19.788459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:19.789179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:19.789543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:19.791628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:19.803209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 544/1070] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:19.809032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:19.908725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42c3df-0145-4b5b-9b11-acbb760109bf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:19.941465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:19.942253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:19.942536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:19.942815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:19.950309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:19.951782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:13:19.966159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:19.967195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:19.968205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 527/1071] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:13:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:19.973097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:20.022470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:20.070340 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a45fc0-85ae-45d8-930b-a70a92e8750c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:20.103721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:20.104378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:20.104657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:20.104930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:20.105765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:20.106070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:20.107513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:20.124275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 545/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:13:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:20.129764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:20.161518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:20.231100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39cf4-5ab1-41bc-a3e6-28496a082e5c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:20.264795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:20.265587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:20.265872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:20.266146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:20.276469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:20.277908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:20.294979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:20.296082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:20.297009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:20.297009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:20.297009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:20.297009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:20.297009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 528/1073] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:20.301785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:20.401250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d416b-3a3a-43cd-9aea-c42302113530 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:20.410493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:20.435810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:20.436474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:20.436752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:20.437027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:20.437743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:20.438050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:20.439441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:20.451315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 546/1074] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:20.456732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:20.542016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:20.557766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d80fa4-79de-4275-acce-d7bc7d8d64b4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:20.590489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:20.591279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:20.591589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:20.591862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:20.598331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:20.599798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:13:20.614578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:20.615716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:20.621333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 529/1075] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:20.621860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:20.729056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad2298-f034-4e8a-8376-9a6f8be0fc7b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:20.763035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880a60a-9e39-4d7b-8004-da9192131b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:20.763705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880a60a-9e39-4d7b-8004-da9192131b59 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:20.763988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880a60a-9e39-4d7b-8004-da9192131b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:20.764256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880a60a-9e39-4d7b-8004-da9192131b59 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:20.765037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880a60a-9e39-4d7b-8004-da9192131b59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:20.765461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880a60a-9e39-4d7b-8004-da9192131b59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:20.766885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880a60a-9e39-4d7b-8004-da9192131b59 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:20.779677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 547/1076] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:13:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:20.785503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:20.856196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294804a0-1266-44d9-ad92-621b4130660d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:20.856784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294804a0-1266-44d9-ad92-621b4130660d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:20.857243 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294804a0-1266-44d9-ad92-621b4130660d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:20.859907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 548/1077] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:20.865014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc963d00-7a02-4a87-be48-d0a08c043846 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:20.865298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc963d00-7a02-4a87-be48-d0a08c043846 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:20.865573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc963d00-7a02-4a87-be48-d0a08c043846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:20.882336 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dc963d00-7a02-4a87-be48-d0a08c043846 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:20.883557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dc963d00-7a02-4a87-be48-d0a08c043846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:20.890806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab152d1-82e2-470d-a84a-c56d34f5387a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:20.906003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:20.924175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2e2dd-16ff-43af-8ed5-e2df8ec36e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:20.925052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2e2dd-16ff-43af-8ed5-e2df8ec36e89 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:20.925328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2e2dd-16ff-43af-8ed5-e2df8ec36e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:20.925599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2e2dd-16ff-43af-8ed5-e2df8ec36e89 None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/5b9888d2731f4117b5fc2c3a720931c4/roles/883cfd8ec8cd40b583eea092a23386af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:20.927794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2e2dd-16ff-43af-8ed5-e2df8ec36e89 None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=5b9888d2731f4117b5fc2c3a720931c4, role_id=883cfd8ec8cd40b583eea092a23386af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:20.929203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2e2dd-16ff-43af-8ed5-e2df8ec36e89 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:20.942324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 530/1078] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:13:20 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/5b9888d2731f4117b5fc2c3a720931c4/roles/883cfd8ec8cd40b583eea092a23386af => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:13:20.951842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 549/1079] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:20.975240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2e2dd-16ff-43af-8ed5-e2df8ec36e89 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:20.979636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:21.080529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2e2dd-16ff-43af-8ed5-e2df8ec36e89 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:21.115272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7d531-5363-4a46-a378-ea23f939f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:21.116099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7d531-5363-4a46-a378-ea23f939f44e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:21.116376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7d531-5363-4a46-a378-ea23f939f44e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:21.116648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7d531-5363-4a46-a378-ea23f939f44e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:21.128934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7d531-5363-4a46-a378-ea23f939f44e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b4d7d531-5363-4a46-a378-ea23f939f44e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:21.137357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:21.138367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:21.139241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 531/1080] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:13:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:21.144768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc963d00-7a02-4a87-be48-d0a08c043846 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:21.242693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc963d00-7a02-4a87-be48-d0a08c043846 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:21.277287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:21.277964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:21.278248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:21.278527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:21.279284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:21.279625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:21.281007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:21.289075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 550/1081] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:13:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:21.295437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7d531-5363-4a46-a378-ea23f939f44e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:21.356974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:21.395806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7d531-5363-4a46-a378-ea23f939f44e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:21.429313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336de678-ebc6-444b-8d4f-7ca6c71ff797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:21.430197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336de678-ebc6-444b-8d4f-7ca6c71ff797 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:21.430998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336de678-ebc6-444b-8d4f-7ca6c71ff797 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:21.431271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336de678-ebc6-444b-8d4f-7ca6c71ff797 None admin] PATH_INFO: `/v3/users/9a19c784d2094bfe99cb13fa37ad5e7b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:21.432497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336de678-ebc6-444b-8d4f-7ca6c71ff797 None admin] RBAC: Authorizing `identity:get_user(user_id=9a19c784d2094bfe99cb13fa37ad5e7b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:21.433924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336de678-ebc6-444b-8d4f-7ca6c71ff797 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:21.435892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 532/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:21 2022] GET /identity/v3/users/9a19c784d2094bfe99cb13fa37ad5e7b => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:21.441330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:21.464891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:21.544579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e857bd3-ec75-499b-b4de-ad390f0ab923 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:21.577675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f682f-73a8-4f22-9ed8-f62e35a87a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:21.578536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f682f-73a8-4f22-9ed8-f62e35a87a97 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:21.578813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f682f-73a8-4f22-9ed8-f62e35a87a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:21.579081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f682f-73a8-4f22-9ed8-f62e35a87a97 None admin] PATH_INFO: `/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:21.580277 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f682f-73a8-4f22-9ed8-f62e35a87a97 None admin] RBAC: Authorizing `identity:get_project(project_id=4a472dd1d1dd4620a18afe26cb3afd58)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:21.581711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f682f-73a8-4f22-9ed8-f62e35a87a97 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:21.583422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 551/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:21 2022] GET /identity/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:21.589900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336de678-ebc6-444b-8d4f-7ca6c71ff797 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:21.687987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336de678-ebc6-444b-8d4f-7ca6c71ff797 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:21.720321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:21.724289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9a5ff-77b2-4fd0-ad2b-d714a634bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:21.725378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9a5ff-77b2-4fd0-ad2b-d714a634bf28 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:21.725660 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9a5ff-77b2-4fd0-ad2b-d714a634bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:21.725934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9a5ff-77b2-4fd0-ad2b-d714a634bf28 None admin] PATH_INFO: `/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58/users/9a19c784d2094bfe99cb13fa37ad5e7b/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:21.727956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9a5ff-77b2-4fd0-ad2b-d714a634bf28 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a472dd1d1dd4620a18afe26cb3afd58, user_id=9a19c784d2094bfe99cb13fa37ad5e7b, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:21.729650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9a5ff-77b2-4fd0-ad2b-d714a634bf28 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:21.744886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 533/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:13:21 2022] PUT /identity/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58/users/9a19c784d2094bfe99cb13fa37ad5e7b/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:13:23.132955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825a7b9-b08a-4e47-9f86-58b86d4b1862 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:23.133459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825a7b9-b08a-4e47-9f86-58b86d4b1862 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:23.133459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825a7b9-b08a-4e47-9f86-58b86d4b1862 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:23.135832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 552/1085] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:23.141086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ec046-84b9-48e4-aa97-c920806c9416 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:23.141504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ec046-84b9-48e4-aa97-c920806c9416 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:23.141638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ec046-84b9-48e4-aa97-c920806c9416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:23.158508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-986ec046-84b9-48e4-aa97-c920806c9416 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:23.159699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-986ec046-84b9-48e4-aa97-c920806c9416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:23.223306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 534/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:23.246195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825a7b9-b08a-4e47-9f86-58b86d4b1862 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:23.334270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:23.349646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825a7b9-b08a-4e47-9f86-58b86d4b1862 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:23.386068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b183cd-630d-41d2-9cc3-b42a1e75c72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:23.386801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b183cd-630d-41d2-9cc3-b42a1e75c72c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:23.387081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b183cd-630d-41d2-9cc3-b42a1e75c72c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:23.387372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b183cd-630d-41d2-9cc3-b42a1e75c72c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:23.397624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b183cd-630d-41d2-9cc3-b42a1e75c72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f8b183cd-630d-41d2-9cc3-b42a1e75c72c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:23.405835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:23.406964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:23.407873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 553/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 17:13:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:23.412926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ec046-84b9-48e4-aa97-c920806c9416 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:23.513928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ec046-84b9-48e4-aa97-c920806c9416 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:23.536068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:23.548876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:23.549664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:23.549953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:23.550239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:23.550991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:23.551302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:23.552689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:23.561399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 535/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 17:13:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:23.567186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b183cd-630d-41d2-9cc3-b42a1e75c72c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:23.682368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b183cd-630d-41d2-9cc3-b42a1e75c72c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:23.716258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:23.717067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:23.717351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:23.717628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:23.729044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:23.730504 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:23.739253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:23.750902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:23.752165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:23.753124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:23.753124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:23.753124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:23.753124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:23.753124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 554/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:23.758366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:23.858815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf688d65-5fc2-4469-b49d-3c7b33818d13 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:23.892681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acbab9-784c-40e5-b9f7-1227484bfca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:23.893359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acbab9-784c-40e5-b9f7-1227484bfca7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:23.893643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acbab9-784c-40e5-b9f7-1227484bfca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:23.893919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acbab9-784c-40e5-b9f7-1227484bfca7 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:23.894643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acbab9-784c-40e5-b9f7-1227484bfca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:23.894952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acbab9-784c-40e5-b9f7-1227484bfca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:23.896341 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acbab9-784c-40e5-b9f7-1227484bfca7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:23.908503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 536/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:23.914118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:23.941159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59edc9-3992-44eb-aabb-b33fad9fb48f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:23.941380 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59edc9-3992-44eb-aabb-b33fad9fb48f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:23.941664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59edc9-3992-44eb-aabb-b33fad9fb48f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:23.943023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 537/1091] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:13:23.948055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150be6e-a363-42c5-b69c-a65df5ea35d0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:23.948343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150be6e-a363-42c5-b69c-a65df5ea35d0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:23.948622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150be6e-a363-42c5-b69c-a65df5ea35d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:23.964827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c150be6e-a363-42c5-b69c-a65df5ea35d0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:23.966045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c150be6e-a363-42c5-b69c-a65df5ea35d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:24.006406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.015778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6c845-67df-45ce-b749-d4f8b057667c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.032896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 538/1092] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:24.048822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.049612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.049895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.050167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.056201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150be6e-a363-42c5-b69c-a65df5ea35d0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.057797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.059242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:13:24.074133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:24.075268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:24.076532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 555/1093] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 17:13:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:24.082988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.141385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.154399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150be6e-a363-42c5-b69c-a65df5ea35d0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.182568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e20a34-a957-488e-a92b-aaea83858881 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.188025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f29f37-9310-4d1d-96c9-182a1b70ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.188818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29f37-9310-4d1d-96c9-182a1b70ac4e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.189106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29f37-9310-4d1d-96c9-182a1b70ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.189379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29f37-9310-4d1d-96c9-182a1b70ac4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.199493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f29f37-9310-4d1d-96c9-182a1b70ac4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-74f29f37-9310-4d1d-96c9-182a1b70ac4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:24.207032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:24.208140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:24.209024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 539/1094] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:24.214617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f29f37-9310-4d1d-96c9-182a1b70ac4e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.215794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.216461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.216742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.217018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.217860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:24.218166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.219621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.237449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 556/1095] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:24.242946 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.252269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.317879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f29f37-9310-4d1d-96c9-182a1b70ac4e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.341712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29de6b-ec2a-4728-9138-721ccef1db01 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.350466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.351130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.351435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.351712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.353242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:24.353633 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.354980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.359364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.363431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 540/1096] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:24.369545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.379686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f26930-4ac4-4358-87c1-afd9caeda895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.380341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f26930-4ac4-4358-87c1-afd9caeda895 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.380626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f26930-4ac4-4358-87c1-afd9caeda895 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.380904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f26930-4ac4-4358-87c1-afd9caeda895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.448436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f26930-4ac4-4358-87c1-afd9caeda895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.449948 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f26930-4ac4-4358-87c1-afd9caeda895 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.463327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.469501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e115-2117-4431-86f8-b7b88d0109a2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.503048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a22ecd-24b1-42e3-b915-71a490a21c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.503897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a22ecd-24b1-42e3-b915-71a490a21c70 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.504187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a22ecd-24b1-42e3-b915-71a490a21c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.504454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a22ecd-24b1-42e3-b915-71a490a21c70 None admin] PATH_INFO: `/v3/users/9a19c784d2094bfe99cb13fa37ad5e7b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.505710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a22ecd-24b1-42e3-b915-71a490a21c70 None admin] RBAC: Authorizing `identity:get_user(user_id=9a19c784d2094bfe99cb13fa37ad5e7b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.507071 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a22ecd-24b1-42e3-b915-71a490a21c70 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.509618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 541/1097] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/users/9a19c784d2094bfe99cb13fa37ad5e7b => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:24.514847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a22ecd-24b1-42e3-b915-71a490a21c70 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.549778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 557/1098] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 17:13:24.555484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f26930-4ac4-4358-87c1-afd9caeda895 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.573369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.613713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a22ecd-24b1-42e3-b915-71a490a21c70 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.646804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d0e37-ccec-4e78-a80e-8e40ba028633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.647485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0e37-ccec-4e78-a80e-8e40ba028633 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.647775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0e37-ccec-4e78-a80e-8e40ba028633 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.648051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0e37-ccec-4e78-a80e-8e40ba028633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.654571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f26930-4ac4-4358-87c1-afd9caeda895 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.678172 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.688082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.688874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.689151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.689422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.699685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.701176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:24.718652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:24.719789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:24.720900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:13:24.720900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:24.720900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:13:24.720900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:24.722384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d0e37-ccec-4e78-a80e-8e40ba028633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.723901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d0e37-ccec-4e78-a80e-8e40ba028633 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.724665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 558/1099] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:24.725433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.799049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.822624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 542/1100] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 17:13:24.828419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d0e37-ccec-4e78-a80e-8e40ba028633 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.830783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489b6e7-d566-4754-8eb7-39f8a9d6cdc5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.866972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.867665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.868476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.868750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.869471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:24.869776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.871148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.882045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 559/1101] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:24.888471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.890429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:24.929132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d0e37-ccec-4e78-a80e-8e40ba028633 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:24.963952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc04df-ae33-4951-b325-ccbfbaf2e5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:24.964745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc04df-ae33-4951-b325-ccbfbaf2e5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:24.965022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc04df-ae33-4951-b325-ccbfbaf2e5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:24.965297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc04df-ae33-4951-b325-ccbfbaf2e5b5 None admin] PATH_INFO: `/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:24.966476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc04df-ae33-4951-b325-ccbfbaf2e5b5 None admin] RBAC: Authorizing `identity:get_project(project_id=4a472dd1d1dd4620a18afe26cb3afd58)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:24.967945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc04df-ae33-4951-b325-ccbfbaf2e5b5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:24.969640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 543/1102] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:24.975574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc04df-ae33-4951-b325-ccbfbaf2e5b5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:24.991936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc8575-ed78-4aec-9b58-1876d997a376 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:25.020481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:25.027114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:25.027935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:25.028217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:25.028486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:25.035190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:25.036669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:13:25.050613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:25.051561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:25.052475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 560/1103] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:25.058460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:25.076611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc04df-ae33-4951-b325-ccbfbaf2e5b5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:25.106177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:25.111910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:25.112557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:25.112833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:25.113105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:25.113930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=4a472dd1d1dd4620a18afe26cb3afd58, user.id=9a19c784d2094bfe99cb13fa37ad5e7b) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:25.114242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:25.115685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:25.122568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 544/1104] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:24 2022] GET /identity/v3/role_assignments?user.id=9a19c784d2094bfe99cb13fa37ad5e7b&scope.project.id=4a472dd1d1dd4620a18afe26cb3afd58&role.id=e07cc5009f624eb9912a837c934e68aa => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:25.161879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f991d-cca6-4b52-83c7-553c157fe762 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:25.201726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec11a37-c2cb-441e-9c5f-79b00b9ab7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:25.202404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11a37-c2cb-441e-9c5f-79b00b9ab7ec None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:25.202683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11a37-c2cb-441e-9c5f-79b00b9ab7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:25.202950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11a37-c2cb-441e-9c5f-79b00b9ab7ec None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:25.203771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec11a37-c2cb-441e-9c5f-79b00b9ab7ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:25.204076 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec11a37-c2cb-441e-9c5f-79b00b9ab7ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:25.205547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec11a37-c2cb-441e-9c5f-79b00b9ab7ec None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:25.219210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 561/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:13:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:13:25.225390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:25.331908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf7298-5478-4a2c-b2db-2c012ff12f7b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:25.366263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f6031-cf7d-4f3b-8648-8c49da2896d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:25.366924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6031-cf7d-4f3b-8648-8c49da2896d5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:25.367202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6031-cf7d-4f3b-8648-8c49da2896d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:25.367505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6031-cf7d-4f3b-8648-8c49da2896d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:25.369529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6031-cf7d-4f3b-8648-8c49da2896d5 None admin] RBAC: Adding query filter params (role.id=883cfd8ec8cd40b583eea092a23386af, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=5b9888d2731f4117b5fc2c3a720931c4) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:25.369851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6031-cf7d-4f3b-8648-8c49da2896d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:25.371275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6031-cf7d-4f3b-8648-8c49da2896d5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:25.377055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 545/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:25 2022] GET /identity/v3/role_assignments?user.id=5b9888d2731f4117b5fc2c3a720931c4&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=883cfd8ec8cd40b583eea092a23386af => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:27.370213 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7226d-bb11-4e2b-a747-f517cc827814 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:27.370750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7226d-bb11-4e2b-a747-f517cc827814 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:27.370750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7226d-bb11-4e2b-a747-f517cc827814 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:27.372173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 562/1107] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:13:27.377270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340aa5-fad3-449b-9291-0abf57717a89 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:27.377661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340aa5-fad3-449b-9291-0abf57717a89 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:27.377821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340aa5-fad3-449b-9291-0abf57717a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:27.394104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-61340aa5-fad3-449b-9291-0abf57717a89 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:27.395264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-61340aa5-fad3-449b-9291-0abf57717a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:27.448133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:27.460916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 546/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:27.488739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7226d-bb11-4e2b-a747-f517cc827814 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:27.538485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:27.588230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7226d-bb11-4e2b-a747-f517cc827814 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:27.619979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b579c07-bb40-490b-8624-552725788d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:27.620803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b579c07-bb40-490b-8624-552725788d1d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:27.621028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b579c07-bb40-490b-8624-552725788d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:27.621297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b579c07-bb40-490b-8624-552725788d1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:27.622442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b579c07-bb40-490b-8624-552725788d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:27.623892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b579c07-bb40-490b-8624-552725788d1d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:27.625473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 563/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:27.631099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61340aa5-fad3-449b-9291-0abf57717a89 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:27.728032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61340aa5-fad3-449b-9291-0abf57717a89 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:27.763846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd9403-8d20-4615-9f4c-29f6421b1bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:27.764555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd9403-8d20-4615-9f4c-29f6421b1bde None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:27.764864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd9403-8d20-4615-9f4c-29f6421b1bde None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:27.765188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd9403-8d20-4615-9f4c-29f6421b1bde None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:27.765952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd9403-8d20-4615-9f4c-29f6421b1bde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:27.767476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd9403-8d20-4615-9f4c-29f6421b1bde None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:27.788458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 547/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:13:27 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:13:30.035597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdd9e9-0cb9-405c-ac60-a3371b0493d3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:30.036126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdd9e9-0cb9-405c-ac60-a3371b0493d3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:30.036126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdd9e9-0cb9-405c-ac60-a3371b0493d3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:30.037453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 564/1111] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:30.043718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0698a-1001-46c0-91c7-3a9669cc4260 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:30.044124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0698a-1001-46c0-91c7-3a9669cc4260 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:30.044273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0698a-1001-46c0-91c7-3a9669cc4260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:30.061710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ebb0698a-1001-46c0-91c7-3a9669cc4260 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:30.062880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebb0698a-1001-46c0-91c7-3a9669cc4260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:30.103241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:30.129171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 548/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:30.153131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdd9e9-0cb9-405c-ac60-a3371b0493d3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:30.252383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdd9e9-0cb9-405c-ac60-a3371b0493d3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:30.268338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:30.285160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b840f-dcc3-4733-bbef-606ae7370fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:30.286100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b840f-dcc3-4733-bbef-606ae7370fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:30.286333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b840f-dcc3-4733-bbef-606ae7370fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:30.286608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b840f-dcc3-4733-bbef-606ae7370fd3 None admin] PATH_INFO: `/v3/roles/64cbdf4e69fa400b8b2f8ed13ebe5367` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:30.288091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b840f-dcc3-4733-bbef-606ae7370fd3 None admin] RBAC: Authorizing `identity:get_role(role_id=64cbdf4e69fa400b8b2f8ed13ebe5367)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:30.289459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b840f-dcc3-4733-bbef-606ae7370fd3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:30.290831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 565/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:30 2022] GET /identity/v3/roles/64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:30.297493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0698a-1001-46c0-91c7-3a9669cc4260 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:30.392305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0698a-1001-46c0-91c7-3a9669cc4260 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:30.425652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f0488-a04d-4ac3-8e96-52020b895c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:30.426515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f0488-a04d-4ac3-8e96-52020b895c9c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:30.426794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f0488-a04d-4ac3-8e96-52020b895c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:30.427068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f0488-a04d-4ac3-8e96-52020b895c9c None admin] PATH_INFO: `/v3/users/5a0982b98d434f3dac03ba7dfa0ecd1d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:30.439784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f0488-a04d-4ac3-8e96-52020b895c9c None admin] RBAC: Authorizing `identity:get_user(user_id=5a0982b98d434f3dac03ba7dfa0ecd1d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:30.441217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f0488-a04d-4ac3-8e96-52020b895c9c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:30.442843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 549/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:30 2022] GET /identity/v3/users/5a0982b98d434f3dac03ba7dfa0ecd1d => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:30.448654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b840f-dcc3-4733-bbef-606ae7370fd3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:30.547130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b840f-dcc3-4733-bbef-606ae7370fd3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:30.579464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535c5c-6702-4d53-beed-63a6f5b7453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:30.580109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535c5c-6702-4d53-beed-63a6f5b7453a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:30.580388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535c5c-6702-4d53-beed-63a6f5b7453a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:30.580662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535c5c-6702-4d53-beed-63a6f5b7453a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:30.632629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:30.646497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34535c5c-6702-4d53-beed-63a6f5b7453a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:30.647964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34535c5c-6702-4d53-beed-63a6f5b7453a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:30.744133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 566/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:13:30.750075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f0488-a04d-4ac3-8e96-52020b895c9c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:30.784106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:30.851842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f0488-a04d-4ac3-8e96-52020b895c9c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:30.888425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c918a-039a-48c2-87bd-dc1083274073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:30.889283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c918a-039a-48c2-87bd-dc1083274073 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:30.889565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c918a-039a-48c2-87bd-dc1083274073 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:30.889838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c918a-039a-48c2-87bd-dc1083274073 None admin] PATH_INFO: `/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:30.891011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c918a-039a-48c2-87bd-dc1083274073 None admin] RBAC: Authorizing `identity:get_project(project_id=4a472dd1d1dd4620a18afe26cb3afd58)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:30.892467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c918a-039a-48c2-87bd-dc1083274073 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:30.894269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 550/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:30 2022] GET /identity/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:30.900326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535c5c-6702-4d53-beed-63a6f5b7453a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:30.982287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:30.998153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535c5c-6702-4d53-beed-63a6f5b7453a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:31.030880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2cf46-1cf6-47b2-a087-24b26fd4201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:31.031565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2cf46-1cf6-47b2-a087-24b26fd4201d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:31.031850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2cf46-1cf6-47b2-a087-24b26fd4201d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:31.032127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2cf46-1cf6-47b2-a087-24b26fd4201d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:31.032950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2cf46-1cf6-47b2-a087-24b26fd4201d None admin] RBAC: Adding query filter params (role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=4a472dd1d1dd4620a18afe26cb3afd58, user.id=5a0982b98d434f3dac03ba7dfa0ecd1d) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:31.033262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2cf46-1cf6-47b2-a087-24b26fd4201d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:31.034702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2cf46-1cf6-47b2-a087-24b26fd4201d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:31.040016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 567/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:30 2022] GET /identity/v3/role_assignments?user.id=5a0982b98d434f3dac03ba7dfa0ecd1d&scope.project.id=4a472dd1d1dd4620a18afe26cb3afd58&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:33.197964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b460d-c9db-40aa-bfe6-187e248e0149 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:33.198480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b460d-c9db-40aa-bfe6-187e248e0149 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:33.198590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b460d-c9db-40aa-bfe6-187e248e0149 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:33.199908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 551/1118] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:33.205265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332347a-3716-4d6d-b897-07a2b2da8f2b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:33.205646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332347a-3716-4d6d-b897-07a2b2da8f2b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:33.205829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332347a-3716-4d6d-b897-07a2b2da8f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:33.223304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5332347a-3716-4d6d-b897-07a2b2da8f2b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:33.224526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5332347a-3716-4d6d-b897-07a2b2da8f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:33.289683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 568/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:33.312488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b460d-c9db-40aa-bfe6-187e248e0149 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:33.410248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b460d-c9db-40aa-bfe6-187e248e0149 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:33.411761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:33.444770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08c1fc-c0d9-4e55-b123-5c55f1c1be9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:33.445623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c1fc-c0d9-4e55-b123-5c55f1c1be9e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:33.445903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c1fc-c0d9-4e55-b123-5c55f1c1be9e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:33.446174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c1fc-c0d9-4e55-b123-5c55f1c1be9e None admin] PATH_INFO: `/v3/roles/64cbdf4e69fa400b8b2f8ed13ebe5367` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:33.448440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08c1fc-c0d9-4e55-b123-5c55f1c1be9e None admin] RBAC: Authorizing `identity:get_role(role_id=64cbdf4e69fa400b8b2f8ed13ebe5367)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:33.449774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08c1fc-c0d9-4e55-b123-5c55f1c1be9e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:33.451144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 552/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:33 2022] GET /identity/v3/roles/64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:33.456855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332347a-3716-4d6d-b897-07a2b2da8f2b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:33.534286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:33.553315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332347a-3716-4d6d-b897-07a2b2da8f2b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:33.585735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe6a88-7886-4b32-8873-49d2ef657d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:33.586589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe6a88-7886-4b32-8873-49d2ef657d29 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:33.586864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe6a88-7886-4b32-8873-49d2ef657d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:33.587133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe6a88-7886-4b32-8873-49d2ef657d29 None admin] PATH_INFO: `/v3/users/5a0982b98d434f3dac03ba7dfa0ecd1d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:33.588343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe6a88-7886-4b32-8873-49d2ef657d29 None admin] RBAC: Authorizing `identity:get_user(user_id=5a0982b98d434f3dac03ba7dfa0ecd1d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:33.589764 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe6a88-7886-4b32-8873-49d2ef657d29 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:33.591378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 569/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:33 2022] GET /identity/v3/users/5a0982b98d434f3dac03ba7dfa0ecd1d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:33.596966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08c1fc-c0d9-4e55-b123-5c55f1c1be9e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:33.694586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08c1fc-c0d9-4e55-b123-5c55f1c1be9e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:33.729024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b35c47-a409-43ad-ab4c-d13e6426438c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:33.729887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35c47-a409-43ad-ab4c-d13e6426438c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:33.730164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35c47-a409-43ad-ab4c-d13e6426438c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:33.730436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35c47-a409-43ad-ab4c-d13e6426438c None admin] PATH_INFO: `/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:33.731625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b35c47-a409-43ad-ab4c-d13e6426438c None admin] RBAC: Authorizing `identity:get_project(project_id=4a472dd1d1dd4620a18afe26cb3afd58)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:33.733040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b35c47-a409-43ad-ab4c-d13e6426438c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:33.734722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 553/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:33 2022] GET /identity/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:33.741006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe6a88-7886-4b32-8873-49d2ef657d29 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:33.840360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe6a88-7886-4b32-8873-49d2ef657d29 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:33.873227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302da61-4a9e-4744-b962-10ab2a91eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:33.874105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302da61-4a9e-4744-b962-10ab2a91eaa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:33.874387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302da61-4a9e-4744-b962-10ab2a91eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:33.874659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302da61-4a9e-4744-b962-10ab2a91eaa3 None admin] PATH_INFO: `/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58/users/5a0982b98d434f3dac03ba7dfa0ecd1d/roles/64cbdf4e69fa400b8b2f8ed13ebe5367` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:33.876890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302da61-4a9e-4744-b962-10ab2a91eaa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a472dd1d1dd4620a18afe26cb3afd58, user_id=5a0982b98d434f3dac03ba7dfa0ecd1d, role_id=64cbdf4e69fa400b8b2f8ed13ebe5367)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:33.878317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302da61-4a9e-4744-b962-10ab2a91eaa3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:33.891323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 570/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:13:33 2022] PUT /identity/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58/users/5a0982b98d434f3dac03ba7dfa0ecd1d/roles/64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:13:36.031054 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fcf2-0316-4566-8fe3-d14177512060 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:36.031609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fcf2-0316-4566-8fe3-d14177512060 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:36.031609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fcf2-0316-4566-8fe3-d14177512060 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:36.032913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 554/1124] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:36.038139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4cf4-bdfc-4743-9a4c-56be5dce3c9f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:36.038575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4cf4-bdfc-4743-9a4c-56be5dce3c9f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:36.038699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4cf4-bdfc-4743-9a4c-56be5dce3c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:36.055692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f7be4cf4-bdfc-4743-9a4c-56be5dce3c9f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:36.057538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f7be4cf4-bdfc-4743-9a4c-56be5dce3c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:36.059483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:36.131969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 571/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:36.156284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397fcf2-0316-4566-8fe3-d14177512060 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:36.199797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:36.259161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397fcf2-0316-4566-8fe3-d14177512060 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:36.292362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5c22a-620d-4e54-b6a9-431f4097bcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:36.293153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5c22a-620d-4e54-b6a9-431f4097bcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:36.293434 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5c22a-620d-4e54-b6a9-431f4097bcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:36.293710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5c22a-620d-4e54-b6a9-431f4097bcc6 None admin] PATH_INFO: `/v3/roles/64cbdf4e69fa400b8b2f8ed13ebe5367` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:36.295181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5c22a-620d-4e54-b6a9-431f4097bcc6 None admin] RBAC: Authorizing `identity:get_role(role_id=64cbdf4e69fa400b8b2f8ed13ebe5367)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:36.296573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5c22a-620d-4e54-b6a9-431f4097bcc6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:36.297932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 555/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:36 2022] GET /identity/v3/roles/64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:36.303837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be4cf4-bdfc-4743-9a4c-56be5dce3c9f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:36.397674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be4cf4-bdfc-4743-9a4c-56be5dce3c9f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:36.431264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400d440-55bc-4930-9fd1-cd122c472c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:36.432540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400d440-55bc-4930-9fd1-cd122c472c81 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:36.432816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400d440-55bc-4930-9fd1-cd122c472c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:36.433086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400d440-55bc-4930-9fd1-cd122c472c81 None admin] PATH_INFO: `/v3/users/5a0982b98d434f3dac03ba7dfa0ecd1d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:36.434265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400d440-55bc-4930-9fd1-cd122c472c81 None admin] RBAC: Authorizing `identity:get_user(user_id=5a0982b98d434f3dac03ba7dfa0ecd1d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:36.435715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400d440-55bc-4930-9fd1-cd122c472c81 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:36.437308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 572/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:36 2022] GET /identity/v3/users/5a0982b98d434f3dac03ba7dfa0ecd1d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:36.442965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5c22a-620d-4e54-b6a9-431f4097bcc6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:36.540462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5c22a-620d-4e54-b6a9-431f4097bcc6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:36.548396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:36.574801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e39318-4f2c-4488-834c-0fb2523fd625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:36.575462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e39318-4f2c-4488-834c-0fb2523fd625 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:36.575745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e39318-4f2c-4488-834c-0fb2523fd625 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:36.576020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e39318-4f2c-4488-834c-0fb2523fd625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:36.640416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e39318-4f2c-4488-834c-0fb2523fd625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:36.641856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e39318-4f2c-4488-834c-0fb2523fd625 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:36.737750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 556/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:13:36.743533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400d440-55bc-4930-9fd1-cd122c472c81 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:36.752775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:36.840033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400d440-55bc-4930-9fd1-cd122c472c81 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:36.872719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27543a-f6c1-4e8d-9a9f-0ec69b033b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:36.873571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27543a-f6c1-4e8d-9a9f-0ec69b033b16 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:36.873850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27543a-f6c1-4e8d-9a9f-0ec69b033b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:36.874123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27543a-f6c1-4e8d-9a9f-0ec69b033b16 None admin] PATH_INFO: `/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:36.875287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27543a-f6c1-4e8d-9a9f-0ec69b033b16 None admin] RBAC: Authorizing `identity:get_project(project_id=4a472dd1d1dd4620a18afe26cb3afd58)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:36.876741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27543a-f6c1-4e8d-9a9f-0ec69b033b16 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:36.878419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 573/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:36 2022] GET /identity/v3/projects/4a472dd1d1dd4620a18afe26cb3afd58 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:36.885284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e39318-4f2c-4488-834c-0fb2523fd625 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:36.893488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:36.983449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e39318-4f2c-4488-834c-0fb2523fd625 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:37.016547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876e8f6-afbf-450e-9567-a0dff4131de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:37.017191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e8f6-afbf-450e-9567-a0dff4131de2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:37.017474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e8f6-afbf-450e-9567-a0dff4131de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:37.017748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e8f6-afbf-450e-9567-a0dff4131de2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:37.018573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e8f6-afbf-450e-9567-a0dff4131de2 None admin] RBAC: Adding query filter params (role.id=64cbdf4e69fa400b8b2f8ed13ebe5367, scope.project.id=4a472dd1d1dd4620a18afe26cb3afd58, user.id=5a0982b98d434f3dac03ba7dfa0ecd1d) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:37.018882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e8f6-afbf-450e-9567-a0dff4131de2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:37.020317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e8f6-afbf-450e-9567-a0dff4131de2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:37.026139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 557/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:36 2022] GET /identity/v3/role_assignments?user.id=5a0982b98d434f3dac03ba7dfa0ecd1d&scope.project.id=4a472dd1d1dd4620a18afe26cb3afd58&role.id=64cbdf4e69fa400b8b2f8ed13ebe5367 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:39.259466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a20c-066c-4bd1-959b-8e57eb3e4c94 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:39.260516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a20c-066c-4bd1-959b-8e57eb3e4c94 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:39.260662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a20c-066c-4bd1-959b-8e57eb3e4c94 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:39.262046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 574/1131] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:39.267382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df265147-ed96-4ca0-ab98-0e88c245d212 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:39.267765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df265147-ed96-4ca0-ab98-0e88c245d212 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:39.267941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df265147-ed96-4ca0-ab98-0e88c245d212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:39.284215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-df265147-ed96-4ca0-ab98-0e88c245d212 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:39.285384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-df265147-ed96-4ca0-ab98-0e88c245d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:39.344713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:39.349682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 558/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:39.373448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7a20c-066c-4bd1-959b-8e57eb3e4c94 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:39.448579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:39.474682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7a20c-066c-4bd1-959b-8e57eb3e4c94 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:39.508110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33717323-ee4b-4a78-b2ca-eda8f2f4d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:39.508885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717323-ee4b-4a78-b2ca-eda8f2f4d088 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:39.509165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717323-ee4b-4a78-b2ca-eda8f2f4d088 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:39.509436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717323-ee4b-4a78-b2ca-eda8f2f4d088 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:39.510654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33717323-ee4b-4a78-b2ca-eda8f2f4d088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:39.512066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33717323-ee4b-4a78-b2ca-eda8f2f4d088 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:39.513660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 575/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:39.519419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df265147-ed96-4ca0-ab98-0e88c245d212 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:39.615319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df265147-ed96-4ca0-ab98-0e88c245d212 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:39.648312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e90ad5-2924-414c-953a-2ae36ebacea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:39.648967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90ad5-2924-414c-953a-2ae36ebacea1 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:39.649249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90ad5-2924-414c-953a-2ae36ebacea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:39.649522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90ad5-2924-414c-953a-2ae36ebacea1 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:39.650260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e90ad5-2924-414c-953a-2ae36ebacea1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:39.651698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e90ad5-2924-414c-953a-2ae36ebacea1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:39.675977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 559/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:13:39 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:13:41.873622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5c9dd-0cbe-4b34-b441-5f57dae2092a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:41.874188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5c9dd-0cbe-4b34-b441-5f57dae2092a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:41.874328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5c9dd-0cbe-4b34-b441-5f57dae2092a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:41.876086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 576/1135] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:41.881442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418cadc-f98c-4307-b20c-ae7778ab5cf5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:41.881908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418cadc-f98c-4307-b20c-ae7778ab5cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:41.882030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418cadc-f98c-4307-b20c-ae7778ab5cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:41.898340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3418cadc-f98c-4307-b20c-ae7778ab5cf5 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:41.899560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3418cadc-f98c-4307-b20c-ae7778ab5cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:41.945394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:41.966953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 560/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:41.991722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5c9dd-0cbe-4b34-b441-5f57dae2092a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:42.096743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5c9dd-0cbe-4b34-b441-5f57dae2092a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:42.131659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecfaa2-782f-434a-9593-4a804c49de06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:42.132447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecfaa2-782f-434a-9593-4a804c49de06 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:42.132724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecfaa2-782f-434a-9593-4a804c49de06 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:42.132992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecfaa2-782f-434a-9593-4a804c49de06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:42.135391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecfaa2-782f-434a-9593-4a804c49de06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:42.136643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecfaa2-782f-434a-9593-4a804c49de06 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:42.138269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 577/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:42.144347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418cadc-f98c-4307-b20c-ae7778ab5cf5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:42.252448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418cadc-f98c-4307-b20c-ae7778ab5cf5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:42.291140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed1154-b403-4b61-9754-0bcb6ce2f62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:42.291917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed1154-b403-4b61-9754-0bcb6ce2f62c None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:42.292197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed1154-b403-4b61-9754-0bcb6ce2f62c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:42.292469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed1154-b403-4b61-9754-0bcb6ce2f62c None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:42.293180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed1154-b403-4b61-9754-0bcb6ce2f62c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:42.294602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed1154-b403-4b61-9754-0bcb6ce2f62c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:42.316409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 561/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:13:42 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:13:44.581944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dcc2e-b88c-4921-9041-c5846da67d64 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:44.582512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dcc2e-b88c-4921-9041-c5846da67d64 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:44.582512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dcc2e-b88c-4921-9041-c5846da67d64 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:44.583892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 578/1139] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:44.589251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a65299-af0d-47dd-8bc5-be31ac3771fc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:44.589657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a65299-af0d-47dd-8bc5-be31ac3771fc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:44.589807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a65299-af0d-47dd-8bc5-be31ac3771fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:44.606873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d3a65299-af0d-47dd-8bc5-be31ac3771fc None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:44.608086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d3a65299-af0d-47dd-8bc5-be31ac3771fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:44.637994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:44.674295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 562/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:13:44.698122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dcc2e-b88c-4921-9041-c5846da67d64 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:44.710051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:44.800631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dcc2e-b88c-4921-9041-c5846da67d64 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:44.833761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3136b-c0bc-46d9-9bfb-1be22bd14283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:44.834475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3136b-c0bc-46d9-9bfb-1be22bd14283 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:44.834754 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3136b-c0bc-46d9-9bfb-1be22bd14283 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:44.835024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3136b-c0bc-46d9-9bfb-1be22bd14283 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:44.844988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3136b-c0bc-46d9-9bfb-1be22bd14283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d1d3136b-c0bc-46d9-9bfb-1be22bd14283 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:44.852505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:44.854170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:44.855301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 579/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:13:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:44.860441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a65299-af0d-47dd-8bc5-be31ac3771fc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:44.956619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a65299-af0d-47dd-8bc5-be31ac3771fc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:44.990486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:44.991145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:44.991460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:44.991740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:44.992481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:44.992787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:44.994866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:45.002776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 563/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:13:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:45.008474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3136b-c0bc-46d9-9bfb-1be22bd14283 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:45.078670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:45.111053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3136b-c0bc-46d9-9bfb-1be22bd14283 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:45.147075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11672c-171d-4617-b5f8-b1b77ad4e76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:45.150710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11672c-171d-4617-b5f8-b1b77ad4e76b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:45.151440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11672c-171d-4617-b5f8-b1b77ad4e76b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:45.151728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11672c-171d-4617-b5f8-b1b77ad4e76b None admin] PATH_INFO: `/v3/users/8591e503f29b4a5897954b60c6b3f9ac` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:45.168210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11672c-171d-4617-b5f8-b1b77ad4e76b None admin] RBAC: Authorizing `identity:get_user(user_id=8591e503f29b4a5897954b60c6b3f9ac)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:45.169651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11672c-171d-4617-b5f8-b1b77ad4e76b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:45.171315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 580/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:45 2022] GET /identity/v3/users/8591e503f29b4a5897954b60c6b3f9ac => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:45.176989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:45.207228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:45.276039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369308d9-44bd-446c-9c0f-585af4a0806d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:45.308700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5fd58-450d-4af1-b90f-1d7b8c5db3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:45.309336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5fd58-450d-4af1-b90f-1d7b8c5db3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:45.309617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5fd58-450d-4af1-b90f-1d7b8c5db3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:45.309892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5fd58-450d-4af1-b90f-1d7b8c5db3f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:45.374946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5fd58-450d-4af1-b90f-1d7b8c5db3f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:45.376340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5fd58-450d-4af1-b90f-1d7b8c5db3f1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:45.404946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:45.472943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 564/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:13:45.479573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11672c-171d-4617-b5f8-b1b77ad4e76b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:45.577282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11672c-171d-4617-b5f8-b1b77ad4e76b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:45.603438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:45.612181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269727-e440-4d10-852b-07139086c1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:45.613034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269727-e440-4d10-852b-07139086c1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:45.613317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269727-e440-4d10-852b-07139086c1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:45.613595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269727-e440-4d10-852b-07139086c1e1 None admin] PATH_INFO: `/v3/projects/163d674b35224eb4a3cee69f14a091b6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:45.614767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b269727-e440-4d10-852b-07139086c1e1 None admin] RBAC: Authorizing `identity:get_project(project_id=163d674b35224eb4a3cee69f14a091b6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:45.616309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b269727-e440-4d10-852b-07139086c1e1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:45.618001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 581/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:45 2022] GET /identity/v3/projects/163d674b35224eb4a3cee69f14a091b6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:45.624165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5fd58-450d-4af1-b90f-1d7b8c5db3f1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:45.722120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5fd58-450d-4af1-b90f-1d7b8c5db3f1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:45.750217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:45.756356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff0d3f-da62-4865-9f44-d2d7b7ada6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:45.757001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff0d3f-da62-4865-9f44-d2d7b7ada6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:45.757280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff0d3f-da62-4865-9f44-d2d7b7ada6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:45.757550 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff0d3f-da62-4865-9f44-d2d7b7ada6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:45.758371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff0d3f-da62-4865-9f44-d2d7b7ada6b0 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=163d674b35224eb4a3cee69f14a091b6, user.id=8591e503f29b4a5897954b60c6b3f9ac) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:45.758681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff0d3f-da62-4865-9f44-d2d7b7ada6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:45.761098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff0d3f-da62-4865-9f44-d2d7b7ada6b0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:45.766364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 565/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:45 2022] GET /identity/v3/role_assignments?user.id=8591e503f29b4a5897954b60c6b3f9ac&scope.project.id=163d674b35224eb4a3cee69f14a091b6&role.id=e07cc5009f624eb9912a837c934e68aa => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:47.958527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddfdc5-5cd8-4b6b-983d-70d105ce3762 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:47.959071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddfdc5-5cd8-4b6b-983d-70d105ce3762 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:47.959071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddfdc5-5cd8-4b6b-983d-70d105ce3762 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:47.960456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 582/1147] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:47.965731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015a98-2aba-4d88-a4b4-1e683cadf56d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:47.966153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015a98-2aba-4d88-a4b4-1e683cadf56d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:47.966284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015a98-2aba-4d88-a4b4-1e683cadf56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:47.983300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-91015a98-2aba-4d88-a4b4-1e683cadf56d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:47.984443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-91015a98-2aba-4d88-a4b4-1e683cadf56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:48.047728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 566/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:48.069736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddfdc5-5cd8-4b6b-983d-70d105ce3762 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:48.168372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddfdc5-5cd8-4b6b-983d-70d105ce3762 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:48.201191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee5353-3e0d-437c-85ad-4a326926133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:48.201970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee5353-3e0d-437c-85ad-4a326926133d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:48.202247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee5353-3e0d-437c-85ad-4a326926133d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:48.202518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee5353-3e0d-437c-85ad-4a326926133d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:48.212187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee5353-3e0d-437c-85ad-4a326926133d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e1ee5353-3e0d-437c-85ad-4a326926133d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:48.220395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:48.221486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:48.222686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 583/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:13:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:13:48.227552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91015a98-2aba-4d88-a4b4-1e683cadf56d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:48.323495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91015a98-2aba-4d88-a4b4-1e683cadf56d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:48.345803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:48.356734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:48.357390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:48.357671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:48.357946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:48.358690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:48.358998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:48.360435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:48.368582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 567/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:13:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:48.374445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee5353-3e0d-437c-85ad-4a326926133d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:48.429554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:48.472567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee5353-3e0d-437c-85ad-4a326926133d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:48.505896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f05925-ad60-4474-a46f-069202f69eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:48.506765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f05925-ad60-4474-a46f-069202f69eee None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:48.507043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f05925-ad60-4474-a46f-069202f69eee None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:48.507315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f05925-ad60-4474-a46f-069202f69eee None admin] PATH_INFO: `/v3/users/8591e503f29b4a5897954b60c6b3f9ac` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:48.508517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f05925-ad60-4474-a46f-069202f69eee None admin] RBAC: Authorizing `identity:get_user(user_id=8591e503f29b4a5897954b60c6b3f9ac)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:48.510661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f05925-ad60-4474-a46f-069202f69eee None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:48.512312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 584/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:48 2022] GET /identity/v3/users/8591e503f29b4a5897954b60c6b3f9ac => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:48.518077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:48.614394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c70609-b86f-48c1-9672-e7ee55807c7c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:48.646467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1781479-e8b4-43f2-a322-ed793b552513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:48.647300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1781479-e8b4-43f2-a322-ed793b552513 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:48.647607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1781479-e8b4-43f2-a322-ed793b552513 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:48.647882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1781479-e8b4-43f2-a322-ed793b552513 None admin] PATH_INFO: `/v3/projects/163d674b35224eb4a3cee69f14a091b6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:48.649043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1781479-e8b4-43f2-a322-ed793b552513 None admin] RBAC: Authorizing `identity:get_project(project_id=163d674b35224eb4a3cee69f14a091b6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:48.651155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1781479-e8b4-43f2-a322-ed793b552513 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:48.652851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 568/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:48 2022] GET /identity/v3/projects/163d674b35224eb4a3cee69f14a091b6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:48.658589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f05925-ad60-4474-a46f-069202f69eee None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:48.755673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f05925-ad60-4474-a46f-069202f69eee None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:48.778632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:48.789060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02f186-2048-4e0b-a204-2c33f9afe061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:48.789947 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02f186-2048-4e0b-a204-2c33f9afe061 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:48.790227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02f186-2048-4e0b-a204-2c33f9afe061 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:48.790503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02f186-2048-4e0b-a204-2c33f9afe061 None admin] PATH_INFO: `/v3/projects/163d674b35224eb4a3cee69f14a091b6/users/8591e503f29b4a5897954b60c6b3f9ac/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:48.792855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02f186-2048-4e0b-a204-2c33f9afe061 None admin] RBAC: Authorizing `identity:create_grant(project_id=163d674b35224eb4a3cee69f14a091b6, user_id=8591e503f29b4a5897954b60c6b3f9ac, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:48.794283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02f186-2048-4e0b-a204-2c33f9afe061 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:48.807018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 585/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:13:48 2022] PUT /identity/v3/projects/163d674b35224eb4a3cee69f14a091b6/users/8591e503f29b4a5897954b60c6b3f9ac/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:13:50.976007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552dea1-1e96-4651-99f3-db0718a27b86 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:50.976560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552dea1-1e96-4651-99f3-db0718a27b86 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:50.976560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552dea1-1e96-4651-99f3-db0718a27b86 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:50.977854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 569/1154] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:50.983193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d5b9-6943-43fd-845c-135f597cbede None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:50.983664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d5b9-6943-43fd-845c-135f597cbede None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:50.983770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d5b9-6943-43fd-845c-135f597cbede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:51.000590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ddd0d5b9-6943-43fd-845c-135f597cbede None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:51.001776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ddd0d5b9-6943-43fd-845c-135f597cbede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:51.066573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 586/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:51.091304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552dea1-1e96-4651-99f3-db0718a27b86 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:51.146613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:51.189839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552dea1-1e96-4651-99f3-db0718a27b86 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:51.222113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadaf9e-0fda-4c3e-9c66-44fff48eecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:51.222889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadaf9e-0fda-4c3e-9c66-44fff48eecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:51.223168 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadaf9e-0fda-4c3e-9c66-44fff48eecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:51.223467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadaf9e-0fda-4c3e-9c66-44fff48eecf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:51.233329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadaf9e-0fda-4c3e-9c66-44fff48eecf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-afadaf9e-0fda-4c3e-9c66-44fff48eecf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:13:51.241173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:51.242354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:13:51.243704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 570/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:51.248673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0d5b9-6943-43fd-845c-135f597cbede None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:51.344128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0d5b9-6943-43fd-845c-135f597cbede None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:51.357594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:51.378697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:51.379371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:51.379659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:51.379935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:51.380683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:51.380990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:51.382357 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:51.390765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 587/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:51.396490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadaf9e-0fda-4c3e-9c66-44fff48eecf4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:51.498351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadaf9e-0fda-4c3e-9c66-44fff48eecf4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:51.511777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:51.531873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebb505-db14-4fcd-a081-3ce36a4cda3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:51.532741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb505-db14-4fcd-a081-3ce36a4cda3c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:51.533023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb505-db14-4fcd-a081-3ce36a4cda3c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:51.533300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb505-db14-4fcd-a081-3ce36a4cda3c None admin] PATH_INFO: `/v3/users/8591e503f29b4a5897954b60c6b3f9ac` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:51.534478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebb505-db14-4fcd-a081-3ce36a4cda3c None admin] RBAC: Authorizing `identity:get_user(user_id=8591e503f29b4a5897954b60c6b3f9ac)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:51.535941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebb505-db14-4fcd-a081-3ce36a4cda3c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:51.537560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 571/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:13:51 2022] GET /identity/v3/users/8591e503f29b4a5897954b60c6b3f9ac => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:51.543845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:51.639447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4e16d-53fa-4238-bb23-c26dbde834b3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:51.671496 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ca3fd-2493-4bd1-a49f-6fb6e27c19e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:51.672143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ca3fd-2493-4bd1-a49f-6fb6e27c19e6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:51.672425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ca3fd-2493-4bd1-a49f-6fb6e27c19e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:51.672700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ca3fd-2493-4bd1-a49f-6fb6e27c19e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:51.717914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:51.741162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ca3fd-2493-4bd1-a49f-6fb6e27c19e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:51.742615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ca3fd-2493-4bd1-a49f-6fb6e27c19e6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:51.840373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 588/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:13:51.846152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebb505-db14-4fcd-a081-3ce36a4cda3c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:51.941628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebb505-db14-4fcd-a081-3ce36a4cda3c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:51.973599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870ea9f-0523-47cd-9fea-2fcbae147fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:51.974446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870ea9f-0523-47cd-9fea-2fcbae147fbe None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:51.974727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870ea9f-0523-47cd-9fea-2fcbae147fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:51.975003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870ea9f-0523-47cd-9fea-2fcbae147fbe None admin] PATH_INFO: `/v3/projects/163d674b35224eb4a3cee69f14a091b6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:51.976189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870ea9f-0523-47cd-9fea-2fcbae147fbe None admin] RBAC: Authorizing `identity:get_project(project_id=163d674b35224eb4a3cee69f14a091b6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:51.977607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870ea9f-0523-47cd-9fea-2fcbae147fbe None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:51.979517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 572/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:13:51 2022] GET /identity/v3/projects/163d674b35224eb4a3cee69f14a091b6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:51.985389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ca3fd-2493-4bd1-a49f-6fb6e27c19e6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:52.059281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:52.082357 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ca3fd-2493-4bd1-a49f-6fb6e27c19e6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:52.114782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532562c-db6e-426c-829a-9b1bdc6a5e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:52.115381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532562c-db6e-426c-829a-9b1bdc6a5e0f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:52.115662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532562c-db6e-426c-829a-9b1bdc6a5e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:52.115933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532562c-db6e-426c-829a-9b1bdc6a5e0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:52.116747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532562c-db6e-426c-829a-9b1bdc6a5e0f None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=163d674b35224eb4a3cee69f14a091b6, user.id=8591e503f29b4a5897954b60c6b3f9ac) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:52.117053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532562c-db6e-426c-829a-9b1bdc6a5e0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:52.118473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532562c-db6e-426c-829a-9b1bdc6a5e0f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:52.124671 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 589/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:13:51 2022] GET /identity/v3/role_assignments?user.id=8591e503f29b4a5897954b60c6b3f9ac&scope.project.id=163d674b35224eb4a3cee69f14a091b6&role.id=e07cc5009f624eb9912a837c934e68aa => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:54.346234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a45102-ab0b-4a5c-a0ff-85dcb880fefa None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:54.346773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a45102-ab0b-4a5c-a0ff-85dcb880fefa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:54.346773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a45102-ab0b-4a5c-a0ff-85dcb880fefa None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:54.348182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 573/1162] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:54.353352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ffe82-4449-4591-b796-56e6f15e74ba None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:54.353787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ffe82-4449-4591-b796-56e6f15e74ba None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:54.353889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ffe82-4449-4591-b796-56e6f15e74ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:54.371931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d39ffe82-4449-4591-b796-56e6f15e74ba None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:54.373114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d39ffe82-4449-4591-b796-56e6f15e74ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:54.444067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 590/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:54.467083 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a45102-ab0b-4a5c-a0ff-85dcb880fefa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:54.517224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:54.566516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a45102-ab0b-4a5c-a0ff-85dcb880fefa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:54.599208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1eb3f-17ff-4522-bd95-d5187dc232a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:54.599809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1eb3f-17ff-4522-bd95-d5187dc232a3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:54.600087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1eb3f-17ff-4522-bd95-d5187dc232a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:54.600364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1eb3f-17ff-4522-bd95-d5187dc232a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:54.664173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1eb3f-17ff-4522-bd95-d5187dc232a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:54.665653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1eb3f-17ff-4522-bd95-d5187dc232a3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:54.710494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:54.760528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 574/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:13:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:13:54.766567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ffe82-4449-4591-b796-56e6f15e74ba None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:54.834035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:54.865657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ffe82-4449-4591-b796-56e6f15e74ba None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:54.897824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca9785-1550-4863-adb6-d3045121911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:54.898616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9785-1550-4863-adb6-d3045121911b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:54.898898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9785-1550-4863-adb6-d3045121911b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:54.899173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9785-1550-4863-adb6-d3045121911b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:54.906392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca9785-1550-4863-adb6-d3045121911b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:54.907875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca9785-1550-4863-adb6-d3045121911b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-20ca9785-1550-4863-adb6-d3045121911b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:13:54.922084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:13:54.923167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:13:54.924130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 591/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:13:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:13:54.929312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1eb3f-17ff-4522-bd95-d5187dc232a3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:55.024325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1eb3f-17ff-4522-bd95-d5187dc232a3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:55.058495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55064ab7-fbfa-42da-89cc-08a1a01577e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:55.059147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55064ab7-fbfa-42da-89cc-08a1a01577e0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:55.059450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55064ab7-fbfa-42da-89cc-08a1a01577e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:55.059720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55064ab7-fbfa-42da-89cc-08a1a01577e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:55.060425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55064ab7-fbfa-42da-89cc-08a1a01577e0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:13:55.060725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55064ab7-fbfa-42da-89cc-08a1a01577e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:55.062066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55064ab7-fbfa-42da-89cc-08a1a01577e0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:55.069003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 575/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 17:13:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:13:57.241893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8483ff-d81e-4427-b554-97490f2cf3f0 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:57.242436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8483ff-d81e-4427-b554-97490f2cf3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:57.242436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8483ff-d81e-4427-b554-97490f2cf3f0 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:57.244583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 592/1167] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:57.249686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67978-e629-4b0e-a305-cd311a5ad143 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:57.250109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67978-e629-4b0e-a305-cd311a5ad143 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:57.250233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67978-e629-4b0e-a305-cd311a5ad143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:57.267113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c9a67978-e629-4b0e-a305-cd311a5ad143 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:57.268329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c9a67978-e629-4b0e-a305-cd311a5ad143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:57.277396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:57.332645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 576/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:57.355600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8483ff-d81e-4427-b554-97490f2cf3f0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:57.453987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8483ff-d81e-4427-b554-97490f2cf3f0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:13:57.481092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:13:57.487155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0ae38-c656-4f90-b874-0dca7ba4de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:13:57.487826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ae38-c656-4f90-b874-0dca7ba4de53 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:57.488104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ae38-c656-4f90-b874-0dca7ba4de53 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:57.488376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ae38-c656-4f90-b874-0dca7ba4de53 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:57.489115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ae38-c656-4f90-b874-0dca7ba4de53 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:13:57.490470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ae38-c656-4f90-b874-0dca7ba4de53 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:13:57.504720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 593/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Sat May 28 17:13:57 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:13:59.774624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb9618-adfe-4f65-a689-84708dc386f1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:59.775118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb9618-adfe-4f65-a689-84708dc386f1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:59.775868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb9618-adfe-4f65-a689-84708dc386f1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:59.777398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 577/1170] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:13:59.782642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3c6bf-5d10-4a9d-a7d4-8baff1eba175 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:13:59.783064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3c6bf-5d10-4a9d-a7d4-8baff1eba175 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:13:59.783198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3c6bf-5d10-4a9d-a7d4-8baff1eba175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:13:59.799554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1cf3c6bf-5d10-4a9d-a7d4-8baff1eba175 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:13:59.800718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1cf3c6bf-5d10-4a9d-a7d4-8baff1eba175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:13:59.867903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 594/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:13:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:13:59.892108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb9618-adfe-4f65-a689-84708dc386f1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:13:59.989873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb9618-adfe-4f65-a689-84708dc386f1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:00.022007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d12be-5220-4efd-a001-9fc5c18fc563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:00.022857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d12be-5220-4efd-a001-9fc5c18fc563 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:00.023137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d12be-5220-4efd-a001-9fc5c18fc563 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:00.023437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d12be-5220-4efd-a001-9fc5c18fc563 None admin] PATH_INFO: `/v3/domains/cbe5e832f55e4b2f8c01ee24bd6b6db4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:00.033525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d12be-5220-4efd-a001-9fc5c18fc563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cbe5e832f55e4b2f8c01ee24bd6b6db4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:00.034951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d12be-5220-4efd-a001-9fc5c18fc563 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:00.036579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 578/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Sat May 28 17:13:59 2022] GET /identity/v3/domains/cbe5e832f55e4b2f8c01ee24bd6b6db4 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:00.042905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3c6bf-5d10-4a9d-a7d4-8baff1eba175 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:00.140959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3c6bf-5d10-4a9d-a7d4-8baff1eba175 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:00.160675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:00.175655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16eccc-e2f1-4601-ae78-1ea40ac1561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:00.176307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16eccc-e2f1-4601-ae78-1ea40ac1561d None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:00.176590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16eccc-e2f1-4601-ae78-1ea40ac1561d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:00.176867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16eccc-e2f1-4601-ae78-1ea40ac1561d None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:00.177580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16eccc-e2f1-4601-ae78-1ea40ac1561d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:00.179020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16eccc-e2f1-4601-ae78-1ea40ac1561d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:00.203703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 595/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:14:00 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:14:02.488117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab214bb-7ddf-40af-99b3-7fd9a15d6472 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:02.488648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab214bb-7ddf-40af-99b3-7fd9a15d6472 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:02.488648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab214bb-7ddf-40af-99b3-7fd9a15d6472 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:02.490044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 579/1174] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:14:02.495757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f52b98-de31-49e7-8760-b38d66959e1c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:02.496202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f52b98-de31-49e7-8760-b38d66959e1c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:02.496297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f52b98-de31-49e7-8760-b38d66959e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:02.513354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d5f52b98-de31-49e7-8760-b38d66959e1c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:14:02.514527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d5f52b98-de31-49e7-8760-b38d66959e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:14:02.579519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 596/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:14:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:14:02.604604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab214bb-7ddf-40af-99b3-7fd9a15d6472 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:02.608317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:02.705960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab214bb-7ddf-40af-99b3-7fd9a15d6472 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:02.739023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e0d85-6365-43a0-8240-c1efaeeba562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:02.739864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e0d85-6365-43a0-8240-c1efaeeba562 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:02.740147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e0d85-6365-43a0-8240-c1efaeeba562 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:02.740425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e0d85-6365-43a0-8240-c1efaeeba562 None admin] PATH_INFO: `/v3/domains/cbe5e832f55e4b2f8c01ee24bd6b6db4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:02.741588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e0d85-6365-43a0-8240-c1efaeeba562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cbe5e832f55e4b2f8c01ee24bd6b6db4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:02.743020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e0d85-6365-43a0-8240-c1efaeeba562 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:02.745226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 580/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Sat May 28 17:14:02 2022] GET /identity/v3/domains/cbe5e832f55e4b2f8c01ee24bd6b6db4 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:02.750870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f52b98-de31-49e7-8760-b38d66959e1c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:02.844752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f52b98-de31-49e7-8760-b38d66959e1c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:02.846242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:02.880304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea769f00-f8db-4354-b024-d9630d22eeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:02.880966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea769f00-f8db-4354-b024-d9630d22eeca None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:02.881249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea769f00-f8db-4354-b024-d9630d22eeca None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:02.881525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea769f00-f8db-4354-b024-d9630d22eeca None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:02.882226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea769f00-f8db-4354-b024-d9630d22eeca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:02.883698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea769f00-f8db-4354-b024-d9630d22eeca None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:02.903011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 597/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:14:02 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:14:05.120833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4542e-e7ef-4e27-af84-72e09ff94b1d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:05.121345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4542e-e7ef-4e27-af84-72e09ff94b1d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:05.121345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4542e-e7ef-4e27-af84-72e09ff94b1d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:05.122669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 581/1178] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:14:05.127940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c9091-dea0-45c0-8d49-29cbbc33cb79 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:05.128375 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c9091-dea0-45c0-8d49-29cbbc33cb79 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:05.128494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c9091-dea0-45c0-8d49-29cbbc33cb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:05.144785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-907c9091-dea0-45c0-8d49-29cbbc33cb79 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:14:05.145972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-907c9091-dea0-45c0-8d49-29cbbc33cb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:14:05.210095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 598/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:14:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:14:05.233146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4542e-e7ef-4e27-af84-72e09ff94b1d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:05.301377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:05.330570 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4542e-e7ef-4e27-af84-72e09ff94b1d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:05.363439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de45bc1-c8d8-4f92-971b-eec7e209740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:05.364218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45bc1-c8d8-4f92-971b-eec7e209740d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:05.364495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45bc1-c8d8-4f92-971b-eec7e209740d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:05.364764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45bc1-c8d8-4f92-971b-eec7e209740d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:05.375492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de45bc1-c8d8-4f92-971b-eec7e209740d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4de45bc1-c8d8-4f92-971b-eec7e209740d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:14:05.382958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:14:05.384105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:14:05.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 582/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:14:05.391064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c9091-dea0-45c0-8d49-29cbbc33cb79 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:05.476434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:05.489154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c9091-dea0-45c0-8d49-29cbbc33cb79 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:05.522366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:05.523029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:05.523310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:05.523637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:05.524381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:14:05.524690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:05.526093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:05.533902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 599/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:14:05.539626 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de45bc1-c8d8-4f92-971b-eec7e209740d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:05.634851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de45bc1-c8d8-4f92-971b-eec7e209740d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:05.662416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:05.668259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababe49b-3396-410e-82a6-663dcd732d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:05.669123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababe49b-3396-410e-82a6-663dcd732d43 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:05.669407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababe49b-3396-410e-82a6-663dcd732d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:05.669685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababe49b-3396-410e-82a6-663dcd732d43 None admin] PATH_INFO: `/v3/users/6209370657a841d7965375424ab52c1b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:05.682516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababe49b-3396-410e-82a6-663dcd732d43 None admin] RBAC: Authorizing `identity:get_user(user_id=6209370657a841d7965375424ab52c1b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:05.684082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababe49b-3396-410e-82a6-663dcd732d43 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:05.685720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 583/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:14:05 2022] GET /identity/v3/users/6209370657a841d7965375424ab52c1b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:05.691361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:05.789189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd81eeb-0247-459e-9d70-0a533bcd9769 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:05.820902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692770ce-a9e1-4a7c-a99a-4e22f9ac4a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:05.821537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692770ce-a9e1-4a7c-a99a-4e22f9ac4a65 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:05.822358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692770ce-a9e1-4a7c-a99a-4e22f9ac4a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:05.822633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692770ce-a9e1-4a7c-a99a-4e22f9ac4a65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:05.840983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:05.888623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692770ce-a9e1-4a7c-a99a-4e22f9ac4a65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:05.890030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692770ce-a9e1-4a7c-a99a-4e22f9ac4a65 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:05.984641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 600/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:14:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:14:05.990741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababe49b-3396-410e-82a6-663dcd732d43 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:06.086812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababe49b-3396-410e-82a6-663dcd732d43 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:06.120697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a676d24-315a-4930-ba00-84e1a7c67095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:06.121543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a676d24-315a-4930-ba00-84e1a7c67095 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:06.122070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a676d24-315a-4930-ba00-84e1a7c67095 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:06.122070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a676d24-315a-4930-ba00-84e1a7c67095 None admin] PATH_INFO: `/v3/projects/e805fb0f40ee46c691fc1409a1bd7c0a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:06.123248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a676d24-315a-4930-ba00-84e1a7c67095 None admin] RBAC: Authorizing `identity:get_project(project_id=e805fb0f40ee46c691fc1409a1bd7c0a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:06.124690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a676d24-315a-4930-ba00-84e1a7c67095 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:06.126416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 584/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:14:05 2022] GET /identity/v3/projects/e805fb0f40ee46c691fc1409a1bd7c0a => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:06.132431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692770ce-a9e1-4a7c-a99a-4e22f9ac4a65 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:06.180944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:06.229713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692770ce-a9e1-4a7c-a99a-4e22f9ac4a65 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:06.263427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfff8d7-7f24-4886-9e6c-bb73e201843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:06.264072 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfff8d7-7f24-4886-9e6c-bb73e201843c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:06.264348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfff8d7-7f24-4886-9e6c-bb73e201843c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:06.264616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfff8d7-7f24-4886-9e6c-bb73e201843c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:06.265434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfff8d7-7f24-4886-9e6c-bb73e201843c None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=e805fb0f40ee46c691fc1409a1bd7c0a, user.id=6209370657a841d7965375424ab52c1b) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:14:06.265740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfff8d7-7f24-4886-9e6c-bb73e201843c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:06.267172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfff8d7-7f24-4886-9e6c-bb73e201843c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:06.272455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 601/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:14:06 2022] GET /identity/v3/role_assignments?user.id=6209370657a841d7965375424ab52c1b&scope.project.id=e805fb0f40ee46c691fc1409a1bd7c0a&role.id=e07cc5009f624eb9912a837c934e68aa => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:08.460411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5458e6-7ed9-4727-898f-510c10d39181 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:08.460980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5458e6-7ed9-4727-898f-510c10d39181 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:08.460980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5458e6-7ed9-4727-898f-510c10d39181 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:08.462332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 585/1186] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:14:08.467689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16161652-faa4-447a-b091-af7ea584798f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:08.468066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16161652-faa4-447a-b091-af7ea584798f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:08.468239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16161652-faa4-447a-b091-af7ea584798f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:08.484554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-16161652-faa4-447a-b091-af7ea584798f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:14:08.485739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-16161652-faa4-447a-b091-af7ea584798f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:14:08.552035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 602/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:14:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:14:08.578273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5458e6-7ed9-4727-898f-510c10d39181 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:08.636533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:08.676330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5458e6-7ed9-4727-898f-510c10d39181 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:08.708886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f042-6d24-46b2-b757-42112a85d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:08.709692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f042-6d24-46b2-b757-42112a85d628 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:08.709970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f042-6d24-46b2-b757-42112a85d628 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:08.710238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f042-6d24-46b2-b757-42112a85d628 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:08.720404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755f042-6d24-46b2-b757-42112a85d628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b755f042-6d24-46b2-b757-42112a85d628 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:14:08.728713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:14:08.729687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:14:08.730669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 586/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:14:08.735854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16161652-faa4-447a-b091-af7ea584798f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:08.780385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:08.832632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16161652-faa4-447a-b091-af7ea584798f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:08.865404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:08.866060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:08.866343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:08.866619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:08.867384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:14:08.867698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:08.869367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:08.877553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 603/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:08.883713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f042-6d24-46b2-b757-42112a85d628 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:08.983862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f042-6d24-46b2-b757-42112a85d628 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:08.999967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:09.016557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff5937-5f8f-4dd2-9944-b4ef88640dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:09.017425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff5937-5f8f-4dd2-9944-b4ef88640dac None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:09.017702 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff5937-5f8f-4dd2-9944-b4ef88640dac None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:09.017970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff5937-5f8f-4dd2-9944-b4ef88640dac None admin] PATH_INFO: `/v3/users/6209370657a841d7965375424ab52c1b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:09.019130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff5937-5f8f-4dd2-9944-b4ef88640dac None admin] RBAC: Authorizing `identity:get_user(user_id=6209370657a841d7965375424ab52c1b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:09.020595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff5937-5f8f-4dd2-9944-b4ef88640dac None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:09.022196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 587/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:14:08 2022] GET /identity/v3/users/6209370657a841d7965375424ab52c1b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:09.028669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:09.129045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb2193-e6a9-4e0d-8795-6a2de9493141 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:09.136463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:09.162491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8ee8f-da24-438d-94df-22a7e1bdcc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:09.163359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8ee8f-da24-438d-94df-22a7e1bdcc9f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:09.163653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8ee8f-da24-438d-94df-22a7e1bdcc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:09.163933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8ee8f-da24-438d-94df-22a7e1bdcc9f None admin] PATH_INFO: `/v3/projects/e805fb0f40ee46c691fc1409a1bd7c0a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:09.165114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8ee8f-da24-438d-94df-22a7e1bdcc9f None admin] RBAC: Authorizing `identity:get_project(project_id=e805fb0f40ee46c691fc1409a1bd7c0a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:09.166756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8ee8f-da24-438d-94df-22a7e1bdcc9f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:09.168473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 604/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:14:09 2022] GET /identity/v3/projects/e805fb0f40ee46c691fc1409a1bd7c0a => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:09.174336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff5937-5f8f-4dd2-9944-b4ef88640dac None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:09.274546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff5937-5f8f-4dd2-9944-b4ef88640dac None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:09.305581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7fb86-d385-432e-88ed-e4a4b0c44906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:09.307005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fb86-d385-432e-88ed-e4a4b0c44906 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:09.307278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fb86-d385-432e-88ed-e4a4b0c44906 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:09.307572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fb86-d385-432e-88ed-e4a4b0c44906 None admin] PATH_INFO: `/v3/projects/e805fb0f40ee46c691fc1409a1bd7c0a/users/6209370657a841d7965375424ab52c1b/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:09.309494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7fb86-d385-432e-88ed-e4a4b0c44906 None admin] RBAC: Authorizing `identity:create_grant(project_id=e805fb0f40ee46c691fc1409a1bd7c0a, user_id=6209370657a841d7965375424ab52c1b, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:09.310894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7fb86-d385-432e-88ed-e4a4b0c44906 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:09.324719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 588/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:14:09 2022] PUT /identity/v3/projects/e805fb0f40ee46c691fc1409a1bd7c0a/users/6209370657a841d7965375424ab52c1b/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:14:11.508553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e824b4-c1d7-4824-9616-4daf47847ed0 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:11.509102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e824b4-c1d7-4824-9616-4daf47847ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:11.509201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e824b4-c1d7-4824-9616-4daf47847ed0 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:11.510728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 605/1193] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:14:11.515943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a98968-0552-4f19-b048-a9e8c5bd4971 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:11.516348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a98968-0552-4f19-b048-a9e8c5bd4971 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:11.516500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a98968-0552-4f19-b048-a9e8c5bd4971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:11.533343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c0a98968-0552-4f19-b048-a9e8c5bd4971 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:14:11.534525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c0a98968-0552-4f19-b048-a9e8c5bd4971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:14:11.575087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:11.600298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 589/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:14:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:14:11.623672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e824b4-c1d7-4824-9616-4daf47847ed0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:11.722281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e824b4-c1d7-4824-9616-4daf47847ed0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:11.755205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038896-cbd3-4514-9318-b6e53b8d219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:11.756015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038896-cbd3-4514-9318-b6e53b8d219c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:11.756292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038896-cbd3-4514-9318-b6e53b8d219c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:11.756559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038896-cbd3-4514-9318-b6e53b8d219c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:11.767625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7038896-cbd3-4514-9318-b6e53b8d219c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f7038896-cbd3-4514-9318-b6e53b8d219c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:14:11.775686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:14:11.776819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:14:11.777727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 606/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:14:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:14:11.782829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a98968-0552-4f19-b048-a9e8c5bd4971 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:11.878747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a98968-0552-4f19-b048-a9e8c5bd4971 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:11.912331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:11.912989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:11.913273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:11.913546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:11.914284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:14:11.914588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:11.916008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:11.923750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 590/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:14:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:11.929481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038896-cbd3-4514-9318-b6e53b8d219c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:11.963325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:12.028002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038896-cbd3-4514-9318-b6e53b8d219c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:12.061148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644606-61cd-47d8-974d-3d35709efafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:12.062016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644606-61cd-47d8-974d-3d35709efafc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:12.062297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644606-61cd-47d8-974d-3d35709efafc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:12.062570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644606-61cd-47d8-974d-3d35709efafc None admin] PATH_INFO: `/v3/users/6209370657a841d7965375424ab52c1b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:12.063786 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a644606-61cd-47d8-974d-3d35709efafc None admin] RBAC: Authorizing `identity:get_user(user_id=6209370657a841d7965375424ab52c1b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:12.065225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a644606-61cd-47d8-974d-3d35709efafc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:12.066845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 607/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 17:14:11 2022] GET /identity/v3/users/6209370657a841d7965375424ab52c1b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:12.073111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:12.112139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:12.169744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96a888-bc20-4353-af49-5b78161acaf4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:12.202508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f901f7d-3c79-4f06-9b95-b563342d4e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:12.203144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901f7d-3c79-4f06-9b95-b563342d4e12 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:12.203459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901f7d-3c79-4f06-9b95-b563342d4e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:12.203738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901f7d-3c79-4f06-9b95-b563342d4e12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:12.267940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f901f7d-3c79-4f06-9b95-b563342d4e12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:12.269366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f901f7d-3c79-4f06-9b95-b563342d4e12 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:12.305587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:12.363979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 591/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:14:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:14:12.370327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644606-61cd-47d8-974d-3d35709efafc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:12.466410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644606-61cd-47d8-974d-3d35709efafc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:12.467940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:12.501602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd3914-20ed-4861-a6bb-61afad311539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:12.502446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd3914-20ed-4861-a6bb-61afad311539 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:12.502724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd3914-20ed-4861-a6bb-61afad311539 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:12.502997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd3914-20ed-4861-a6bb-61afad311539 None admin] PATH_INFO: `/v3/projects/e805fb0f40ee46c691fc1409a1bd7c0a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:12.504192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd3914-20ed-4861-a6bb-61afad311539 None admin] RBAC: Authorizing `identity:get_project(project_id=e805fb0f40ee46c691fc1409a1bd7c0a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:12.505614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd3914-20ed-4861-a6bb-61afad311539 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:12.507279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 608/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 17:14:12 2022] GET /identity/v3/projects/e805fb0f40ee46c691fc1409a1bd7c0a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:12.513281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f901f7d-3c79-4f06-9b95-b563342d4e12 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:14:12.609099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f901f7d-3c79-4f06-9b95-b563342d4e12 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:14:12.642043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b2619-7424-4081-b6a2-59a217558dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:14:12.642684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2619-7424-4081-b6a2-59a217558dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:14:12.642962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2619-7424-4081-b6a2-59a217558dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:14:12.643234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2619-7424-4081-b6a2-59a217558dc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:14:12.644088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b2619-7424-4081-b6a2-59a217558dc7 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=e805fb0f40ee46c691fc1409a1bd7c0a, user.id=6209370657a841d7965375424ab52c1b) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:14:12.644393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b2619-7424-4081-b6a2-59a217558dc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:14:12.645797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b2619-7424-4081-b6a2-59a217558dc7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:14:12.651551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 592/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:14:12 2022] GET /identity/v3/role_assignments?user.id=6209370657a841d7965375424ab52c1b&scope.project.id=e805fb0f40ee46c691fc1409a1bd7c0a&role.id=e07cc5009f624eb9912a837c934e68aa => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:14:22.497540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=15 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:14:22.648900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:12.596964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b58c7-076e-46f8-b2a7-fc7259d8733d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:12.597632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b58c7-076e-46f8-b2a7-fc7259d8733d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:12.597632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b58c7-076e-46f8-b2a7-fc7259d8733d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:12.598904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 609/1201] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:15:12.605040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858b92b-810a-4b31-9093-557342d2cf4d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:12.605680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858b92b-810a-4b31-9093-557342d2cf4d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:12.605680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858b92b-810a-4b31-9093-557342d2cf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:12.631820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3858b92b-810a-4b31-9093-557342d2cf4d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:12.633772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3858b92b-810a-4b31-9093-557342d2cf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:12.731107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 593/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:12.761319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b58c7-076e-46f8-b2a7-fc7259d8733d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:12.900712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b58c7-076e-46f8-b2a7-fc7259d8733d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:12.951054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:12.952512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:12.952801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:12.953159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:12.966276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:12.970524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:12.987904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:12.989350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:12.990335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:12.990335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:12.990335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:12.990335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:12.992297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 610/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:15:13.000910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858b92b-810a-4b31-9093-557342d2cf4d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:13.138526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858b92b-810a-4b31-9093-557342d2cf4d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:13.178738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:13.182695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a8838-3d78-44b1-b5f3-402209f92084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:13.183417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a8838-3d78-44b1-b5f3-402209f92084 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:13.183699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a8838-3d78-44b1-b5f3-402209f92084 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:13.183980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a8838-3d78-44b1-b5f3-402209f92084 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:13.184715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a8838-3d78-44b1-b5f3-402209f92084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:13.185026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a8838-3d78-44b1-b5f3-402209f92084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:13.186401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a8838-3d78-44b1-b5f3-402209f92084 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:13.209656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:13.210078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 594/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:13.285171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:13.344638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f097b-f351-4e3b-bc8b-597d8b175bc1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:13.387579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94995873-2dec-4ff6-9b41-284d6d7bd113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:13.388362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94995873-2dec-4ff6-9b41-284d6d7bd113 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:13.388659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94995873-2dec-4ff6-9b41-284d6d7bd113 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:13.388939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94995873-2dec-4ff6-9b41-284d6d7bd113 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:13.389781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94995873-2dec-4ff6-9b41-284d6d7bd113 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:13.391513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94995873-2dec-4ff6-9b41-284d6d7bd113 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:13.423428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 611/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 17:15:13 2022] POST /identity/v3/users => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:15:16.047073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64f417-16a6-454f-b99b-6f2ba912da96 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:16.047609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64f417-16a6-454f-b99b-6f2ba912da96 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:16.047609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64f417-16a6-454f-b99b-6f2ba912da96 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:16.049091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 595/1206] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:16.054770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758e079-9005-497a-a987-a7242a589fd3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:16.055176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758e079-9005-497a-a987-a7242a589fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:16.055377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758e079-9005-497a-a987-a7242a589fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:16.081313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d758e079-9005-497a-a987-a7242a589fd3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:16.083741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d758e079-9005-497a-a987-a7242a589fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:16.162937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 612/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:16.189873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64f417-16a6-454f-b99b-6f2ba912da96 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:16.307681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64f417-16a6-454f-b99b-6f2ba912da96 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:16.353212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8be10-ee08-41b3-8f50-11073df3ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:16.354363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8be10-ee08-41b3-8f50-11073df3ab48 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:16.354651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8be10-ee08-41b3-8f50-11073df3ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:16.355603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8be10-ee08-41b3-8f50-11073df3ab48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:16.368360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8be10-ee08-41b3-8f50-11073df3ab48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-26b8be10-ee08-41b3-8f50-11073df3ab48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:15:16.379494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:16.380548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:16.381416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 596/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:16.388192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758e079-9005-497a-a987-a7242a589fd3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:16.434616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:16.517658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758e079-9005-497a-a987-a7242a589fd3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:16.553708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:16.554377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:16.554653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:16.554922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:16.556301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:16.556809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:16.558588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:16.567939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 613/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:15:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:16.574178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8be10-ee08-41b3-8f50-11073df3ab48 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:16.632896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:16.699357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8be10-ee08-41b3-8f50-11073df3ab48 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:16.741269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:16.742065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:16.742351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:16.742642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:16.754993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:16.756498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:16.778453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:16.779515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:16.780438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:16.780438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:16.780438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:16.780438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:16.784465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 597/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:16.785353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:16.897921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:16.901825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bbbee-e40c-41fe-ae77-127d2eb74440 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:16.942413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:16.943086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:16.943413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:16.943811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:16.944411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:16.944718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:16.946061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:16.963534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 614/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:16.965344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:17.093630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45684a24-e9cc-42df-95ed-a4e865719b72 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:17.095304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:17.136450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:17.137343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:17.137651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:17.137930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:17.146373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:17.147858 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:15:17.165732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:17.166760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:17.167836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 598/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:17.174425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:17.281941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd41e74-3cfb-4343-aa61-31b64375c59a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:17.319945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:17.320711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:17.321137 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:17.322037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:17.323035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:17.323386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:17.324866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:17.347066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 615/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 17:15:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:17.352908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:17.481028 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4038a9f-7d27-4d79-a558-1b24c2a71978 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:17.531758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208648e-7292-4782-8512-d533be5bbc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:17.537176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208648e-7292-4782-8512-d533be5bbc45 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:17.537176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208648e-7292-4782-8512-d533be5bbc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:17.537176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208648e-7292-4782-8512-d533be5bbc45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:17.568916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:17.617420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208648e-7292-4782-8512-d533be5bbc45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:17.619014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208648e-7292-4782-8512-d533be5bbc45 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:17.750165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 599/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:15:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:15:17.751898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:17.775454 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:17.870928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa73e76-d8ab-4d1f-9950-b768b0c634ae None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:17.913701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:17.914801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:17.915906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:17.916375 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:17.928381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:17.931646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:17.953133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:17.954292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:17.960348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:17.960348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:17.960348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:17.960348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:17.960348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 616/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:17.960805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208648e-7292-4782-8512-d533be5bbc45 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:18.033301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:18.075951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208648e-7292-4782-8512-d533be5bbc45 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:18.111617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:18.112290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:18.112560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:18.112837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:18.113825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:18.114131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:18.115568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:18.132797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 600/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:18.133608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:18.218497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:18.253942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276ab9-ece2-46e3-94fd-058b57bfae47 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:18.292395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:18.293515 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:18.293816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:18.294109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:18.301662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:18.303126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:15:18.318147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:18.319171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:18.325186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 617/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:18.325741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:18.448933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd8fe-a66e-4442-b435-2869c4c69da1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:18.481060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:18.498554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4711a76-83c5-4329-b93b-55ab5b102195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:18.499244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711a76-83c5-4329-b93b-55ab5b102195 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:18.499713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711a76-83c5-4329-b93b-55ab5b102195 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:18.500098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711a76-83c5-4329-b93b-55ab5b102195 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:18.500976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4711a76-83c5-4329-b93b-55ab5b102195 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:18.501441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4711a76-83c5-4329-b93b-55ab5b102195 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:18.503093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4711a76-83c5-4329-b93b-55ab5b102195 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:18.522960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 601/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:18.529930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:18.658567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce9fc-33db-44ae-9e9c-e1c3cbce7adf None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:18.693371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:18.700329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0836a-ed90-46e4-87e1-35ab7de2fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:18.701259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0836a-ed90-46e4-87e1-35ab7de2fdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:18.701500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0836a-ed90-46e4-87e1-35ab7de2fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:18.701771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0836a-ed90-46e4-87e1-35ab7de2fdb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:18.702617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0836a-ed90-46e4-87e1-35ab7de2fdb6 None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=9d2bfa61bb7245579090eb060baf69ce) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:18.702937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0836a-ed90-46e4-87e1-35ab7de2fdb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:18.704464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0836a-ed90-46e4-87e1-35ab7de2fdb6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:18.710835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 618/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:15:18 2022] GET /identity/v3/role_assignments?user.id=9d2bfa61bb7245579090eb060baf69ce&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:20.973978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d117b-740d-4b87-9ddc-7f39c5117da4 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:20.974506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d117b-740d-4b87-9ddc-7f39c5117da4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:20.974506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d117b-740d-4b87-9ddc-7f39c5117da4 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:20.975998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 602/1220] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:20.981785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e933d-3df5-4629-b738-10a4f84d60ea None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:20.982220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e933d-3df5-4629-b738-10a4f84d60ea None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:20.982414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e933d-3df5-4629-b738-10a4f84d60ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:21.001463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ff8e933d-3df5-4629-b738-10a4f84d60ea None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:21.002671 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ff8e933d-3df5-4629-b738-10a4f84d60ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:21.076508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 619/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:21.100135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d117b-740d-4b87-9ddc-7f39c5117da4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:21.235407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d117b-740d-4b87-9ddc-7f39c5117da4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:21.282191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadcfec-7aa8-437f-9e19-f03533db23ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:21.283001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadcfec-7aa8-437f-9e19-f03533db23ba None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:21.283284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadcfec-7aa8-437f-9e19-f03533db23ba None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:21.283586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadcfec-7aa8-437f-9e19-f03533db23ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:21.295400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beadcfec-7aa8-437f-9e19-f03533db23ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-beadcfec-7aa8-437f-9e19-f03533db23ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:15:21.303621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:21.304735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:15:21.306215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:21.307477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 603/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:15:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:21.314599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e933d-3df5-4629-b738-10a4f84d60ea None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:21.422973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e933d-3df5-4629-b738-10a4f84d60ea None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:21.447684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:21.459511 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:21.460180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:21.460457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:21.460726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:21.461468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:21.461772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:21.463438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:21.472302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 620/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:15:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:21.479891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadcfec-7aa8-437f-9e19-f03533db23ba None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:21.531674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:21.594577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadcfec-7aa8-437f-9e19-f03533db23ba None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:21.630796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:21.631617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:21.631897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:21.632205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:21.644662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:21.646484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:21.664727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:21.666875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:21.668734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:21.668734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:21.668734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:21.668734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:21.668734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 604/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:21.673749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:21.785761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf775-39c1-4658-8fbc-e53db40687b5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:21.824574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:21.825237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:21.825516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:21.825790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:21.826509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:21.826814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:21.828359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:21.844159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 621/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:15:21.850871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:21.970543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:21.979845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab60f8-90e4-4e2a-9f95-5e3562698ad8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:22.017114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:22.018018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:22.018299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:22.018616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:22.027100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:22.028580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:15:22.045997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:22.047212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:22.048392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 605/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:15:22.053494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:22.104500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:22.172450 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88fc0-b1ee-4342-be16-fe8b69085aa6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:22.212550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:22.213409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:22.213727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:22.214077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:22.214962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:22.215386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:22.216993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:22.236326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 622/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 17:15:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:22.242951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:22.357393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ac3e4-40d9-49a0-b1ee-ad296f8e0d6b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:22.393981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:22.394822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:22.395141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:22.395489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:22.409001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:22.410502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:22.428540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:22.429679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:22.431074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:22.431074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:22.431074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:22.431074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:22.434686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 606/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:22.435631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:22.553736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f9565-00a1-468b-82e7-e4750e9992c2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:22.556346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:22.592723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:22.593745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:22.594146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:22.594434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:22.595234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:22.595704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:22.597391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:22.617185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 623/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:22.619037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:22.629903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:22.732706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c19e2-15c6-4dfb-b003-b6a1f72113c9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:22.768762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:22.769560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:22.769843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:22.770118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:22.777327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:22.778926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:15:22.793808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:22.794991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:22.796139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 607/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:22.801531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:22.919669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7175a-3b3f-4582-95dc-09227061e03a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:22.955662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09ca50-bc38-4ec9-a024-104f35d87bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:22.956563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09ca50-bc38-4ec9-a024-104f35d87bfc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:22.956986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09ca50-bc38-4ec9-a024-104f35d87bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:22.957324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09ca50-bc38-4ec9-a024-104f35d87bfc None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:22.958352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09ca50-bc38-4ec9-a024-104f35d87bfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:22.958732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09ca50-bc38-4ec9-a024-104f35d87bfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:22.960284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09ca50-bc38-4ec9-a024-104f35d87bfc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:22.976074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 624/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:22.983279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:23.063719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:23.100801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81747a7-3043-479b-892d-82e0b4f306fe None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:23.141730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0cbec-5012-4c0b-b9bd-4656b04cca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:23.142753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0cbec-5012-4c0b-b9bd-4656b04cca40 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:23.143036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0cbec-5012-4c0b-b9bd-4656b04cca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:23.143311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0cbec-5012-4c0b-b9bd-4656b04cca40 None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/9d2bfa61bb7245579090eb060baf69ce/roles/7dd141eb611c4f98a92d16b8b21b7a9d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:23.159682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0cbec-5012-4c0b-b9bd-4656b04cca40 None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=9d2bfa61bb7245579090eb060baf69ce, role_id=7dd141eb611c4f98a92d16b8b21b7a9d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:23.161602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0cbec-5012-4c0b-b9bd-4656b04cca40 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:23.174928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 608/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:15:22 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/9d2bfa61bb7245579090eb060baf69ce/roles/7dd141eb611c4f98a92d16b8b21b7a9d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:15:25.438389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1d1a-6951-4d80-a1b6-9aba1b494f3e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:25.438951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1d1a-6951-4d80-a1b6-9aba1b494f3e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:25.438951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1d1a-6951-4d80-a1b6-9aba1b494f3e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:25.440354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 625/1233] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:25.445692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a29c0-12a9-4233-8e72-e0548adf981f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:25.446133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a29c0-12a9-4233-8e72-e0548adf981f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:25.446829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a29c0-12a9-4233-8e72-e0548adf981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:25.465021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d17a29c0-12a9-4233-8e72-e0548adf981f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:25.466541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d17a29c0-12a9-4233-8e72-e0548adf981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:25.568033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 609/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:25.592523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da1d1a-6951-4d80-a1b6-9aba1b494f3e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:25.627252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:25.717606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da1d1a-6951-4d80-a1b6-9aba1b494f3e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:25.758247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001145a-2d5b-423e-bfbc-37d94bbb025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:25.759748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001145a-2d5b-423e-bfbc-37d94bbb025e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:25.760031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001145a-2d5b-423e-bfbc-37d94bbb025e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:25.760307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001145a-2d5b-423e-bfbc-37d94bbb025e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:25.772431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001145a-2d5b-423e-bfbc-37d94bbb025e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a001145a-2d5b-423e-bfbc-37d94bbb025e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:15:25.782605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:25.783781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:25.784665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 626/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 17:15:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:15:25.791938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a29c0-12a9-4233-8e72-e0548adf981f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:25.850149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:25.907535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a29c0-12a9-4233-8e72-e0548adf981f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:25.948266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:25.949078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:25.949459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:25.949856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:25.950738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:25.951055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:25.952658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:25.962588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 610/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 17:15:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:25.968478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001145a-2d5b-423e-bfbc-37d94bbb025e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:26.068699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:26.086194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001145a-2d5b-423e-bfbc-37d94bbb025e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:26.126247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:26.127257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:26.127674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:26.128006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:26.146901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:26.148433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:26.170639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:26.171821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:26.173378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:26.173378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:26.173378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:26.173378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:26.173378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 627/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:26.177934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:26.293314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded8360-7bd1-41e5-86b4-569c99bd7af3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:26.305172 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:26.339980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:26.340651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:26.340930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:26.341204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:26.341938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:26.342225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:26.343611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:26.356101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 611/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:26.361768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:26.480241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89c0e0-9599-4ff8-91ac-b1972bb4f22d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:26.520773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:26.522057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:26.522533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:26.522947 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:26.527847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:26.535360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:26.537675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:15:26.557484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:26.558746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:26.560078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 628/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:26.564847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:26.677154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c63301-d0ab-403f-a220-368dbabb1b66 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:26.717209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:26.717891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:26.718175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:26.718451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:26.719306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:26.719647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:26.721156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:26.741764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 612/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 17:15:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:26.748924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:26.873749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386215a1-f81c-4d5f-bbae-8d35fd2a938f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:26.913151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58bea5-74fd-4dde-b962-133be51fcdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:26.913940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bea5-74fd-4dde-b962-133be51fcdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:26.914437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bea5-74fd-4dde-b962-133be51fcdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:26.914799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bea5-74fd-4dde-b962-133be51fcdc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:26.985550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:26.993986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58bea5-74fd-4dde-b962-133be51fcdc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:26.995484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58bea5-74fd-4dde-b962-133be51fcdc6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:27.103502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 629/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:15:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:15:27.109206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:27.142186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:27.225906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17aa7b-212e-4613-927c-34a08142553c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:27.264964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:27.265756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:27.266036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:27.266308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:27.277584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:27.279019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:27.298576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:27.299556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:27.301097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:27.301097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:27.301097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:27.301097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:27.301097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 613/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:27.306563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58bea5-74fd-4dde-b962-133be51fcdc6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:27.409302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:27.418993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58bea5-74fd-4dde-b962-133be51fcdc6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:27.462991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:27.463908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:27.464358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:27.464659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:27.465628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:27.466124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:27.468198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:27.489151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 630/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:27.495268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:27.585008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:27.610595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656dbce-7709-442f-b095-c6f32653fa47 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:27.649179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:27.651137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:27.651476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:27.651817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:27.658817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:27.660334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:15:27.674124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:27.675265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:27.676741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 614/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:15:27.681346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:27.789735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9b669-4709-4328-9b83-393ba94f06a0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:27.826752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e808ce-ad73-4783-a6c1-39311fd71ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:27.827444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e808ce-ad73-4783-a6c1-39311fd71ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:27.827729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e808ce-ad73-4783-a6c1-39311fd71ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:27.828022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e808ce-ad73-4783-a6c1-39311fd71ed2 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:27.828799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e808ce-ad73-4783-a6c1-39311fd71ed2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:27.829110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e808ce-ad73-4783-a6c1-39311fd71ed2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:27.830603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e808ce-ad73-4783-a6c1-39311fd71ed2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:27.846328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 631/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:27.853004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:27.968943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc972-cf08-4ba5-85e8-464a50b4a8c0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:28.005861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b80f58-0c58-48b1-9944-179707e33e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:28.006526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b80f58-0c58-48b1-9944-179707e33e0e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:28.006813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b80f58-0c58-48b1-9944-179707e33e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:28.007152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b80f58-0c58-48b1-9944-179707e33e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:28.008044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b80f58-0c58-48b1-9944-179707e33e0e None admin] RBAC: Adding query filter params (role.id=7dd141eb611c4f98a92d16b8b21b7a9d, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=9d2bfa61bb7245579090eb060baf69ce) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:28.008363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b80f58-0c58-48b1-9944-179707e33e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:28.009801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b80f58-0c58-48b1-9944-179707e33e0e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:28.022105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 615/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:15:27 2022] GET /identity/v3/role_assignments?user.id=9d2bfa61bb7245579090eb060baf69ce&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=7dd141eb611c4f98a92d16b8b21b7a9d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:30.333992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f6587-8dcd-482b-96e3-1c457e3aac2e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:30.334529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f6587-8dcd-482b-96e3-1c457e3aac2e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:30.334529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f6587-8dcd-482b-96e3-1c457e3aac2e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:30.335917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 632/1247] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:30.341978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f15b9-5669-46cc-9c85-cd1a89ab3693 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:30.342520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f15b9-5669-46cc-9c85-cd1a89ab3693 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:30.342520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f15b9-5669-46cc-9c85-cd1a89ab3693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:30.347591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:30.361253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-092f15b9-5669-46cc-9c85-cd1a89ab3693 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:30.362558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-092f15b9-5669-46cc-9c85-cd1a89ab3693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:30.434159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 616/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:30.457274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f6587-8dcd-482b-96e3-1c457e3aac2e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:30.477153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:30.579107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f6587-8dcd-482b-96e3-1c457e3aac2e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:30.615473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced3c71-df0a-4a21-ba18-f531e1c4189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:30.616198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced3c71-df0a-4a21-ba18-f531e1c4189c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:30.616525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced3c71-df0a-4a21-ba18-f531e1c4189c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:30.616797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced3c71-df0a-4a21-ba18-f531e1c4189c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:30.631492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced3c71-df0a-4a21-ba18-f531e1c4189c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-fced3c71-df0a-4a21-ba18-f531e1c4189c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:15:30.639941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:30.640913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:15:30.642122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:30.646062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 633/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:15:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:30.647078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f15b9-5669-46cc-9c85-cd1a89ab3693 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:30.763636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f15b9-5669-46cc-9c85-cd1a89ab3693 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:30.794649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:30.801536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:30.802612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:30.802935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:30.803317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:30.804246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:30.804749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:30.806502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:30.816553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 617/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:15:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:30.822810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced3c71-df0a-4a21-ba18-f531e1c4189c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:30.918212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:30.946824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced3c71-df0a-4a21-ba18-f531e1c4189c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:30.985636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:30.986584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:30.986918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:30.987326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:30.999861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:31.001394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:31.020101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:31.021337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:31.022445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:31.022445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:31.022445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:31.022445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:31.027743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 634/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:15:31.029160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:31.137519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce66d5-44d7-4d5b-90c3-959bca7015bb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:31.173619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:31.174225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:31.174507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:31.174780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:31.175571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:31.175873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:31.177272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:31.189085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 618/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:31.194791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:31.311139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45104dce-1993-4d3e-ada5-d6e09252b87c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:31.341973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:31.351012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:31.352112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:31.352403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:31.352747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:31.361512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:31.363007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:15:31.378484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:31.379736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:31.381010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:31.385487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 635/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 17:15:31.386739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:31.447405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:31.501385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd67e2e-d656-47a2-9829-15e992932f9c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:31.547762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:31.548878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:31.549285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:31.549793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:31.550936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:31.551408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:31.553292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:31.573982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 619/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 17:15:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:15:31.579949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:31.706682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baace0-bac8-42b2-8ba7-556893839750 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:31.749816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cf73d-601e-4ba6-adea-be3805293d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:31.750890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cf73d-601e-4ba6-adea-be3805293d48 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:31.751189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cf73d-601e-4ba6-adea-be3805293d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:31.751622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cf73d-601e-4ba6-adea-be3805293d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:31.816658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:31.831950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cf73d-601e-4ba6-adea-be3805293d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:31.833418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cf73d-601e-4ba6-adea-be3805293d48 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:31.943737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 636/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:15:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 17:15:31.950542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:32.058963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdaec2-9aed-43c4-9ea5-c32b478f19e2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:32.074873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:32.097764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:32.098572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:32.098856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:32.099132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:32.109726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:32.111219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:32.130431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:32.131653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:32.138683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:32.138683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:32.138683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:32.138683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:32.138683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 620/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:32.139039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cf73d-601e-4ba6-adea-be3805293d48 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:32.228877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:32.241465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cf73d-601e-4ba6-adea-be3805293d48 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:32.277321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:32.278002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:32.278280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:32.278552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:32.279285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:32.279620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:32.281054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:32.293955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 637/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:32.299611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:32.403924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4972c-7f8e-42ef-9344-b0b87232cd38 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:32.438834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:32.439659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:32.439948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:32.440224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:32.447615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:32.449056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:15:32.462635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:32.463738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:32.464822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 621/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:15:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:32.470459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:32.577918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe681429-02d4-41d2-9908-6f33be2948e6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:32.611480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961d511-153b-4d22-95ea-9a0b21f2a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:32.612150 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d511-153b-4d22-95ea-9a0b21f2a768 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:32.612432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d511-153b-4d22-95ea-9a0b21f2a768 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:32.612716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d511-153b-4d22-95ea-9a0b21f2a768 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:32.613506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961d511-153b-4d22-95ea-9a0b21f2a768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:32.613817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961d511-153b-4d22-95ea-9a0b21f2a768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:32.615543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961d511-153b-4d22-95ea-9a0b21f2a768 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:32.628564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 638/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:15:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:32.635139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:32.662972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:32.738100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69e6d7-cae4-4aaa-a0b7-2c5edcde4595 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:32.771146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869de7f-a873-4597-9b6a-5a3a9e0dd642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:32.771834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869de7f-a873-4597-9b6a-5a3a9e0dd642 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:32.772120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869de7f-a873-4597-9b6a-5a3a9e0dd642 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:32.772396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869de7f-a873-4597-9b6a-5a3a9e0dd642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:32.773223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869de7f-a873-4597-9b6a-5a3a9e0dd642 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=9d2bfa61bb7245579090eb060baf69ce) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:32.773534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869de7f-a873-4597-9b6a-5a3a9e0dd642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:32.775224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869de7f-a873-4597-9b6a-5a3a9e0dd642 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:32.780545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 622/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:15:32 2022] GET /identity/v3/role_assignments?user.id=9d2bfa61bb7245579090eb060baf69ce&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=e07cc5009f624eb9912a837c934e68aa => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:34.930310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1692-c843-4e6a-91c7-0ccd01ea91b8 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:34.931906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1692-c843-4e6a-91c7-0ccd01ea91b8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:34.931906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1692-c843-4e6a-91c7-0ccd01ea91b8 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:34.932879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 639/1261] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:34.937987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a504a-650e-435e-9912-14241f1f7fdc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:34.938427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a504a-650e-435e-9912-14241f1f7fdc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:34.938546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a504a-650e-435e-9912-14241f1f7fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:34.955724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-912a504a-650e-435e-9912-14241f1f7fdc None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:34.956916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-912a504a-650e-435e-9912-14241f1f7fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:35.021360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 623/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 17:15:35.043730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda1692-c843-4e6a-91c7-0ccd01ea91b8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:35.054334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:35.141602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda1692-c843-4e6a-91c7-0ccd01ea91b8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:35.173799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d63a9b-d5b9-4107-abfe-c0576c10ce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:35.174572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d63a9b-d5b9-4107-abfe-c0576c10ce75 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:35.174849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d63a9b-d5b9-4107-abfe-c0576c10ce75 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:35.175119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d63a9b-d5b9-4107-abfe-c0576c10ce75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:35.185456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d63a9b-d5b9-4107-abfe-c0576c10ce75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-29d63a9b-d5b9-4107-abfe-c0576c10ce75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:15:35.192979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:35.194117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:35.195180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 640/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:15:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:35.200194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a504a-650e-435e-9912-14241f1f7fdc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:35.217663 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:35.296808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a504a-650e-435e-9912-14241f1f7fdc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:35.329761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:35.330426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:35.330712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:35.330989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:35.331770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:35.332079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:35.333436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:35.341976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 624/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:15:35.347824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d63a9b-d5b9-4107-abfe-c0576c10ce75 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:35.413284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:35.447231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d63a9b-d5b9-4107-abfe-c0576c10ce75 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:35.483321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:35.484367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:35.484661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:35.484975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:35.496491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:35.497969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:35.515074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:35.516342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:35.517493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:35.517493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:35.517493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:35.517493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:35.517493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 641/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:35.522113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:35.601921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:35.621186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e1155-014c-4608-b2df-fea66f17c973 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:35.653182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:35.653833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:35.654115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:35.654390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:35.655101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:35.655440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:35.656787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:35.669428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 625/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:15:35.675169 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:35.778654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896275a5-ca36-4317-ab5e-e05a70bc9507 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:35.797532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:35.811366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17752711-164b-42db-a267-9458759dfee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:35.812158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17752711-164b-42db-a267-9458759dfee6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:35.812437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17752711-164b-42db-a267-9458759dfee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:35.812708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17752711-164b-42db-a267-9458759dfee6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:35.819950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17752711-164b-42db-a267-9458759dfee6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:35.821911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17752711-164b-42db-a267-9458759dfee6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-17752711-164b-42db-a267-9458759dfee6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:15:35.839354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:35.840323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:35.841437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 642/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:35.846341 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:35.944188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd1854-5698-4e7d-9ae8-669dc0b32ab9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:35.975932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:35.976589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:35.976868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:35.977141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:35.978511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:35.978820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:35.980280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:35.988471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:35.997815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 626/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 17:15:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:36.003428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17752711-164b-42db-a267-9458759dfee6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:36.100333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17752711-164b-42db-a267-9458759dfee6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:36.133008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:36.133927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:36.134281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:36.134605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:36.144629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:36.146062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:36.163091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:36.164191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:36.165777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:36.165777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:36.165777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:36.165777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:36.165777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 643/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:36.170533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:36.272987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cf035-ce69-48ec-85a5-df1a240f8e48 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:36.306610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:36.307270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:36.307582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:36.307863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:36.308584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:36.308894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:36.310257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:36.321238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 627/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:36.326988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:36.358768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:36.424959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be20eac-42db-4122-9dc5-c59385c1e060 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:36.456828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:36.457606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:36.457888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:36.458163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:36.465149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:36.466589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:15:36.483175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:36.484391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:36.485293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 644/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:36.491789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:36.549304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:36.590273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d67ea-1183-43da-a744-9393d25e8a92 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:36.622458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadebeb-08da-43f1-aefc-82b118111b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:36.623112 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadebeb-08da-43f1-aefc-82b118111b84 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:36.623423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadebeb-08da-43f1-aefc-82b118111b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:36.623705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadebeb-08da-43f1-aefc-82b118111b84 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:36.624487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadebeb-08da-43f1-aefc-82b118111b84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:36.624802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadebeb-08da-43f1-aefc-82b118111b84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:36.626226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadebeb-08da-43f1-aefc-82b118111b84 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:36.638568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 628/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:15:36.644476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:36.734150 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:36.743273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7834a96-bf56-4712-b97b-c754e5cbd1b1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:36.776930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb2de1-5f83-4dc9-a3fb-1e9662c8bc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:36.777812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb2de1-5f83-4dc9-a3fb-1e9662c8bc2d None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:36.778092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb2de1-5f83-4dc9-a3fb-1e9662c8bc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:36.778364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb2de1-5f83-4dc9-a3fb-1e9662c8bc2d None admin] PATH_INFO: `/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/9d2bfa61bb7245579090eb060baf69ce/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:36.780334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb2de1-5f83-4dc9-a3fb-1e9662c8bc2d None admin] RBAC: Authorizing `identity:create_grant(project_id=ceee56bba3764f4ca3ec96f17d12848b, user_id=9d2bfa61bb7245579090eb060baf69ce, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:36.781776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb2de1-5f83-4dc9-a3fb-1e9662c8bc2d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:36.794312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 645/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 17:15:36 2022] PUT /identity/v3/projects/ceee56bba3764f4ca3ec96f17d12848b/users/9d2bfa61bb7245579090eb060baf69ce/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:15:38.954682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154a5c7-c8e7-48ed-bd47-b94aaa65d476 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:38.955207 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154a5c7-c8e7-48ed-bd47-b94aaa65d476 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:38.955294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154a5c7-c8e7-48ed-bd47-b94aaa65d476 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:38.956640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 629/1274] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:38.962478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e74007-9243-4577-bd5d-8b78961a9add None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:38.962874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e74007-9243-4577-bd5d-8b78961a9add None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:38.963028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e74007-9243-4577-bd5d-8b78961a9add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:38.979384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-13e74007-9243-4577-bd5d-8b78961a9add None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:38.980518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-13e74007-9243-4577-bd5d-8b78961a9add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:39.046796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 646/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:39.069858 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154a5c7-c8e7-48ed-bd47-b94aaa65d476 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:39.169639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154a5c7-c8e7-48ed-bd47-b94aaa65d476 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:39.186613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:39.203809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b05692-196c-4680-b844-cf97e8484234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:39.204602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b05692-196c-4680-b844-cf97e8484234 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:39.204883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b05692-196c-4680-b844-cf97e8484234 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:39.205154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b05692-196c-4680-b844-cf97e8484234 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:39.214994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b05692-196c-4680-b844-cf97e8484234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-85b05692-196c-4680-b844-cf97e8484234 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 17:15:39.222551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:39.223690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:39.224930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 630/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 17:15:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:39.229835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e74007-9243-4577-bd5d-8b78961a9add None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:39.308171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:39.327243 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e74007-9243-4577-bd5d-8b78961a9add None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:39.359530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:39.360184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:39.360462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:39.360735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:39.361474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:39.361780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:39.363158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:39.372015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 647/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 17:15:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:39.377627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b05692-196c-4680-b844-cf97e8484234 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:39.475118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b05692-196c-4680-b844-cf97e8484234 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:39.515990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:39.516768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:39.517114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:39.517333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:39.528977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:39.530593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:39.551530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:39.552614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:39.553901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:39.553901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:39.553901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:39.553901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:39.553901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 631/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:39.558612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:39.655787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f5d29-018e-4ea7-a96c-82d3fbc71c41 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:39.687432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:39.688078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:39.688358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:39.688631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:39.689341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:39.689642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:39.691003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:39.702768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 648/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:39.708217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:39.737924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:39.805498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c41b41-381f-4da0-9b02-c1e9c712c717 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:39.838460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:39.839242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:39.839546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:39.839818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:39.846524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:39.847975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 17:15:39.862128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:39.863259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:39.864565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 632/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:39.869881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:39.873654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:39.967814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57e485-986e-482c-9c1b-c024d7f86bf1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:40.001191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:40.001879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:40.002159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:40.002433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:40.003268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:40.003606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:40.005029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:40.022452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 649/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 17:15:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:40.028251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:40.113954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:40.124452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953e58-1a5c-4efc-a97e-1277334047b4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:40.157041 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94338-32ae-4994-b453-c544b542d9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:40.157678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94338-32ae-4994-b453-c544b542d9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:40.157960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94338-32ae-4994-b453-c544b542d9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:40.158233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94338-32ae-4994-b453-c544b542d9e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:40.223380 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b94338-32ae-4994-b453-c544b542d9e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:40.224805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b94338-32ae-4994-b453-c544b542d9e4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:40.316414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:40.326799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 633/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 17:15:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 17:15:40.332597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:40.394579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:40.432809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c096128-0b87-432e-9dbe-6ea5eb28a558 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:40.466254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:40.467035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:40.467399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:40.467713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:40.480683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:40.482348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 17:15:40.502768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:40.503892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:40.504997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 17:15:40.504997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 17:15:40.504997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 17:15:40.504997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:40.504997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 650/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:40.510274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94338-32ae-4994-b453-c544b542d9e4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:40.610947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94338-32ae-4994-b453-c544b542d9e4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:40.644298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:40.644998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:40.645231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:40.645506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:40.646214 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:40.646521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:40.647934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:40.659316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 634/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 17:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:40.665477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:40.763014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be09194-1468-4638-9712-0cd0f0d56440 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:40.777020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:40.797229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:40.798010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:40.798292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:40.798567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:40.805020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:40.806479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 17:15:40.819834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:40.820886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application May 28 17:15:40.822247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 651/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 17:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:40.826830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:40.875718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:40.927303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e2022-b499-4822-ba21-381dfb2b38d1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:40.963330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b7bbb-4dd1-403c-8337-d09c15cf97d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:40.964005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7bbb-4dd1-403c-8337-d09c15cf97d4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:40.964281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7bbb-4dd1-403c-8337-d09c15cf97d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:40.964552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7bbb-4dd1-403c-8337-d09c15cf97d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:40.965323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b7bbb-4dd1-403c-8337-d09c15cf97d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:40.965627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b7bbb-4dd1-403c-8337-d09c15cf97d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:40.967051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b7bbb-4dd1-403c-8337-d09c15cf97d4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:40.979164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 635/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 17:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:40.985581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:41.081897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b56a2-c9da-4df2-a305-0c87dd3e3d97 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:41.115586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba32e1-b4f6-488e-88cf-076def7c9c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:41.116224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba32e1-b4f6-488e-88cf-076def7c9c16 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:41.116502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba32e1-b4f6-488e-88cf-076def7c9c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:41.116775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba32e1-b4f6-488e-88cf-076def7c9c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:41.117595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba32e1-b4f6-488e-88cf-076def7c9c16 None admin] RBAC: Adding query filter params (role.id=e07cc5009f624eb9912a837c934e68aa, scope.project.id=ceee56bba3764f4ca3ec96f17d12848b, user.id=9d2bfa61bb7245579090eb060baf69ce) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:41.117902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba32e1-b4f6-488e-88cf-076def7c9c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:41.119618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba32e1-b4f6-488e-88cf-076def7c9c16 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:41.125269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 652/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 17:15:40 2022] GET /identity/v3/role_assignments?user.id=9d2bfa61bb7245579090eb060baf69ce&scope.project.id=ceee56bba3764f4ca3ec96f17d12848b&role.id=e07cc5009f624eb9912a837c934e68aa => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:43.345601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d8b5-472b-42d7-b21e-48afddab2a35 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:43.346198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d8b5-472b-42d7-b21e-48afddab2a35 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:43.346198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d8b5-472b-42d7-b21e-48afddab2a35 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:43.347540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 636/1288] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:43.352906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b75d-c257-450d-88dc-923d46f6b882 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:43.353312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b75d-c257-450d-88dc-923d46f6b882 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:43.353456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b75d-c257-450d-88dc-923d46f6b882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:43.369948 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-abc9b75d-c257-450d-88dc-923d46f6b882 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:43.371104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-abc9b75d-c257-450d-88dc-923d46f6b882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:43.400252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:43.436317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 653/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:43.459077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984d8b5-472b-42d7-b21e-48afddab2a35 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:43.572562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984d8b5-472b-42d7-b21e-48afddab2a35 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:43.582747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:43.605821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:43.606556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:43.606834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:43.607103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:43.607783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:43.609133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:43.621716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:43.622930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:43.624044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 637/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:15:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:43.628924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9b75d-c257-450d-88dc-923d46f6b882 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:43.724793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9b75d-c257-450d-88dc-923d46f6b882 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:43.758051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56319d-4a87-4560-9fa5-e00faf9fe567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:43.759451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56319d-4a87-4560-9fa5-e00faf9fe567 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:43.759795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56319d-4a87-4560-9fa5-e00faf9fe567 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:43.760070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56319d-4a87-4560-9fa5-e00faf9fe567 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:43.760779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56319d-4a87-4560-9fa5-e00faf9fe567 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:43.761086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56319d-4a87-4560-9fa5-e00faf9fe567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:43.762452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56319d-4a87-4560-9fa5-e00faf9fe567 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:43.767182 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 654/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Sat May 28 17:15:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:43.772812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:43.875629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f87d-b51f-4a39-834a-ada132652163 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:43.910200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbd82b-8e20-4631-8934-5b58e6471786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:43.910864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd82b-8e20-4631-8934-5b58e6471786 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:43.911149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd82b-8e20-4631-8934-5b58e6471786 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:43.911454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd82b-8e20-4631-8934-5b58e6471786 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:43.912158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd82b-8e20-4631-8934-5b58e6471786 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:43.912464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd82b-8e20-4631-8934-5b58e6471786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:43.913822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd82b-8e20-4631-8934-5b58e6471786 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:43.918761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 638/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Sat May 28 17:15:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:46.116693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26804e2-6680-46ba-ae84-dd6b59a61553 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:46.117215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26804e2-6680-46ba-ae84-dd6b59a61553 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:46.117215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26804e2-6680-46ba-ae84-dd6b59a61553 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:46.118558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 655/1293] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:46.123784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13fd6-2168-442c-a6c5-176a39a6eab8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:46.124158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13fd6-2168-442c-a6c5-176a39a6eab8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:46.124337 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13fd6-2168-442c-a6c5-176a39a6eab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:46.140653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d8f13fd6-2168-442c-a6c5-176a39a6eab8 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:46.141826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d8f13fd6-2168-442c-a6c5-176a39a6eab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:46.171365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:46.207648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 639/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:46.235154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26804e2-6680-46ba-ae84-dd6b59a61553 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:46.266482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:46.341138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26804e2-6680-46ba-ae84-dd6b59a61553 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:46.373551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff52e02-6681-4f66-8dd8-04521c2a3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:46.374140 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff52e02-6681-4f66-8dd8-04521c2a3d00 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:46.374417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff52e02-6681-4f66-8dd8-04521c2a3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:46.374688 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff52e02-6681-4f66-8dd8-04521c2a3d00 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:46.375432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff52e02-6681-4f66-8dd8-04521c2a3d00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:46.376796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff52e02-6681-4f66-8dd8-04521c2a3d00 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:46.389796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 656/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 17:15:46 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:15:48.600319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730f1e4-d56d-4867-b9b3-63d1a729db3f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:48.600878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730f1e4-d56d-4867-b9b3-63d1a729db3f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:48.600878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730f1e4-d56d-4867-b9b3-63d1a729db3f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:48.602227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 640/1296] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:48.607631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882cde9-276a-4788-9d47-e434738bccea None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:48.608033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882cde9-276a-4788-9d47-e434738bccea None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:48.608188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882cde9-276a-4788-9d47-e434738bccea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:48.624869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5882cde9-276a-4788-9d47-e434738bccea None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:48.626046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5882cde9-276a-4788-9d47-e434738bccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:48.695742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 657/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:48.718842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730f1e4-d56d-4867-b9b3-63d1a729db3f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:48.817260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730f1e4-d56d-4867-b9b3-63d1a729db3f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:48.849480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:48.850261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:48.850596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:48.850819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:48.851493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:48.853417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:48.865253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:48.866450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:48.867554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 641/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:15:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:48.872555 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882cde9-276a-4788-9d47-e434738bccea None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:48.890359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:48.969700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882cde9-276a-4788-9d47-e434738bccea None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:49.002913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:49.003594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:49.003872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:49.004141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:49.004844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] RBAC: Adding query filter params (name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:49.005148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:49.006527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:49.011564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 658/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Sat May 28 17:15:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:49.017300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:49.053165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:49.121382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea75b0-03d0-4b59-ac63-e74dc54fc659 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:49.154571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd04f7-c602-43c2-8aaf-436509878431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:49.155238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd04f7-c602-43c2-8aaf-436509878431 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:49.155552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd04f7-c602-43c2-8aaf-436509878431 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:49.155834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd04f7-c602-43c2-8aaf-436509878431 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:49.156626 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd04f7-c602-43c2-8aaf-436509878431 None admin] RBAC: Adding query filter params (interface=public, service_id=6a8c92e50879499c9e96db5b34591191, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:49.156942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd04f7-c602-43c2-8aaf-436509878431 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:49.158304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd04f7-c602-43c2-8aaf-436509878431 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:49.163332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 642/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 17:15:49 2022] GET /identity/v3/endpoints?service_id=6a8c92e50879499c9e96db5b34591191&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:49.169427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:49.247789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:49.266170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69333665-25bd-42e5-8d1a-afb47c9d90ee None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:49.299121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fcd11-69ca-4bf0-ad99-29e602370f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:49.299800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fcd11-69ca-4bf0-ad99-29e602370f01 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:49.300082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fcd11-69ca-4bf0-ad99-29e602370f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:49.300355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fcd11-69ca-4bf0-ad99-29e602370f01 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:49.301027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fcd11-69ca-4bf0-ad99-29e602370f01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:49.302426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fcd11-69ca-4bf0-ad99-29e602370f01 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:49.309082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 659/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 17:15:49 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:15:51.492089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78aa9b-9674-4938-958b-7e438c34ab47 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:51.492620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78aa9b-9674-4938-958b-7e438c34ab47 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:51.492620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78aa9b-9674-4938-958b-7e438c34ab47 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:51.494063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 643/1302] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:15:51.499775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd394fce-f583-4d3f-86c9-f0b438d5275f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:51.500258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd394fce-f583-4d3f-86c9-f0b438d5275f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:51.500361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd394fce-f583-4d3f-86c9-f0b438d5275f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:51.518268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bd394fce-f583-4d3f-86c9-f0b438d5275f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:15:51.519477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bd394fce-f583-4d3f-86c9-f0b438d5275f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:15:51.584306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 660/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:15:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 17:15:51.608149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78aa9b-9674-4938-958b-7e438c34ab47 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:51.700486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:51.707724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78aa9b-9674-4938-958b-7e438c34ab47 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:51.745303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:51.746044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:51.746324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:51.746594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:51.747236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:51.748618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 17:15:51.760587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 17:15:51.761856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 28 17:15:51.763004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 644/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 17:15:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 17:15:51.767935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd394fce-f583-4d3f-86c9-f0b438d5275f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:51.842181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:15:51.865498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd394fce-f583-4d3f-86c9-f0b438d5275f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:51.897547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5247ca9-c7ef-47c0-99f7-f3a54d3da3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:51.898202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247ca9-c7ef-47c0-99f7-f3a54d3da3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:51.898483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247ca9-c7ef-47c0-99f7-f3a54d3da3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:51.898755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247ca9-c7ef-47c0-99f7-f3a54d3da3d4 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:51.900051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5247ca9-c7ef-47c0-99f7-f3a54d3da3d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:15:51.900353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5247ca9-c7ef-47c0-99f7-f3a54d3da3d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:51.901725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5247ca9-c7ef-47c0-99f7-f3a54d3da3d4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:51.906714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 661/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Sat May 28 17:15:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:15:51.912493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:15:52.008804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce52cf8-0239-4bf9-999e-07bf3db04014 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:15:52.041874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7150c-bc99-42c5-b318-5634d6c40582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:15:52.042523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7150c-bc99-42c5-b318-5634d6c40582 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:15:52.042810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7150c-bc99-42c5-b318-5634d6c40582 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:15:52.043090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7150c-bc99-42c5-b318-5634d6c40582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:15:52.043874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7150c-bc99-42c5-b318-5634d6c40582 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:15:52.045233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7150c-bc99-42c5-b318-5634d6c40582 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:15:52.063218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 645/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 17:15:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:16:01.904183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=30 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:16:02.059009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:16:55.837621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a79175-b440-4438-bbb0-90d668b3ef78 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:16:55.838053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a79175-b440-4438-bbb0-90d668b3ef78 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:16:55.838053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a79175-b440-4438-bbb0-90d668b3ef78 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:16:55.843368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 662/1307] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:16:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:16:55.844640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ec5a-8920-45db-9c97-8abce9151c1c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:16:55.845101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ec5a-8920-45db-9c97-8abce9151c1c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:16:55.845283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ec5a-8920-45db-9c97-8abce9151c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:16:55.936961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8fd4ec5a-8920-45db-9c97-8abce9151c1c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:16:55.938170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8fd4ec5a-8920-45db-9c97-8abce9151c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:16:56.152193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 646/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:16:56.195734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7b6b-c0fe-4002-966b-cae93082becb None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:16:56.196884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7b6b-c0fe-4002-966b-cae93082becb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:16:56.197399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7b6b-c0fe-4002-966b-cae93082becb None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:16:56.198906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 663/1309] 10.222.0.33 () {54 vars in 964 bytes} [Sat May 28 17:16:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:16:56.205330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d787d-fb96-4e3b-9efd-4b12e51a4018 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:16:56.205775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d787d-fb96-4e3b-9efd-4b12e51a4018 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:16:56.206074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d787d-fb96-4e3b-9efd-4b12e51a4018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:16:56.372103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ef0d787d-fb96-4e3b-9efd-4b12e51a4018 None None] MFA Rules not processed for user `71a48b3b4fa94798b158488f08bae743`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:16:56.373386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ef0d787d-fb96-4e3b-9efd-4b12e51a4018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:16:56.395282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:16:56.476843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 647/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Sat May 28 17:16:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:16:56.487474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca7b6b-c0fe-4002-966b-cae93082becb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:16:56.489609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c4ca7b6b-c0fe-4002-966b-cae93082becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:16:56.612174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca7b6b-c0fe-4002-966b-cae93082becb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:16:56.694116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc639a0-3b30-4274-8298-53d8586909ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:16:56.694777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc639a0-3b30-4274-8298-53d8586909ac service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:16:56.695060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc639a0-3b30-4274-8298-53d8586909ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:16:56.695361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc639a0-3b30-4274-8298-53d8586909ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:16:56.697885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ccc639a0-3b30-4274-8298-53d8586909ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:16:56.760894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:16:56.827268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc639a0-3b30-4274-8298-53d8586909ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:16:56.829159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc639a0-3b30-4274-8298-53d8586909ac service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:16:56.850468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 664/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:16:59.457769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa55c8-3dd4-4c98-8e2c-8a353b326403 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:16:59.458271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa55c8-3dd4-4c98-8e2c-8a353b326403 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:16:59.458353 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa55c8-3dd4-4c98-8e2c-8a353b326403 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:16:59.463972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 648/1312] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:16:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:16:59.465918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d962-2f78-4d01-9413-9a34ac64312c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:16:59.466348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d962-2f78-4d01-9413-9a34ac64312c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:16:59.466482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d962-2f78-4d01-9413-9a34ac64312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:16:59.549850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0dc9d962-2f78-4d01-9413-9a34ac64312c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:16:59.551090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0dc9d962-2f78-4d01-9413-9a34ac64312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:16:59.650823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 665/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:16:59.688522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa55c8-3dd4-4c98-8e2c-8a353b326403 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:16:59.690082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2eaa55c8-3dd4-4c98-8e2c-8a353b326403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:16:59.800174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa55c8-3dd4-4c98-8e2c-8a353b326403 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:16:59.872421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:16:59.874987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146647e-b696-4a2b-9762-1f9e2128d5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:16:59.875721 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146647e-b696-4a2b-9762-1f9e2128d5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:16:59.876184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146647e-b696-4a2b-9762-1f9e2128d5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:16:59.876610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146647e-b696-4a2b-9762-1f9e2128d5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:16:59.878537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6146647e-b696-4a2b-9762-1f9e2128d5b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:16:59.987718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146647e-b696-4a2b-9762-1f9e2128d5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:16:59.989160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146647e-b696-4a2b-9762-1f9e2128d5b7 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:00.007781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 649/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:02.512693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444a18f-0f51-47fc-98d0-f22f571e127c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:02.513233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444a18f-0f51-47fc-98d0-f22f571e127c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:02.513233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444a18f-0f51-47fc-98d0-f22f571e127c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:02.514709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 666/1315] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:02.519921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f783dfc-7207-43a2-b38a-5d720375b1b5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:02.520334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f783dfc-7207-43a2-b38a-5d720375b1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:02.520480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f783dfc-7207-43a2-b38a-5d720375b1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:02.583323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5f783dfc-7207-43a2-b38a-5d720375b1b5 None None] MFA Rules not processed for user `61bb099d65764401bb3305bd6d0703a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:02.584533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5f783dfc-7207-43a2-b38a-5d720375b1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:02.636495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:02.682883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 650/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:02.722180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444a18f-0f51-47fc-98d0-f22f571e127c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:02.723744 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a444a18f-0f51-47fc-98d0-f22f571e127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:02.734262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:02.827105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444a18f-0f51-47fc-98d0-f22f571e127c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:02.899687 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306795b-3ca4-4b51-b61d-cc561c05334e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:02.900329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306795b-3ca4-4b51-b61d-cc561c05334e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:02.900603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306795b-3ca4-4b51-b61d-cc561c05334e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:02.900868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306795b-3ca4-4b51-b61d-cc561c05334e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:02.902535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7306795b-3ca4-4b51-b61d-cc561c05334e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:02.963472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:03.009315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306795b-3ca4-4b51-b61d-cc561c05334e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:03.010743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306795b-3ca4-4b51-b61d-cc561c05334e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:03.028732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 667/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:05.739903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8c192-a34d-4b3f-8e5e-f67dab5e9858 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:05.740364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8c192-a34d-4b3f-8e5e-f67dab5e9858 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:05.740454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8c192-a34d-4b3f-8e5e-f67dab5e9858 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:05.741845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 651/1318] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:17:05.747487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2dc5d-693b-4af5-a2bd-9b713b16f5da None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:05.747894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2dc5d-693b-4af5-a2bd-9b713b16f5da None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:05.748044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2dc5d-693b-4af5-a2bd-9b713b16f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:05.812921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7bb2dc5d-693b-4af5-a2bd-9b713b16f5da None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:05.814134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7bb2dc5d-693b-4af5-a2bd-9b713b16f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:05.908797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 668/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:05.962108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305b8c2-cf37-4298-808d-263be52c9364 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:05.962624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305b8c2-cf37-4298-808d-263be52c9364 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:05.963036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305b8c2-cf37-4298-808d-263be52c9364 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:05.964436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 652/1320] 10.222.0.33 () {54 vars in 964 bytes} [Sat May 28 17:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:05.970521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab56733-ec58-49f8-9a29-78f9cbded75d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:05.970755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab56733-ec58-49f8-9a29-78f9cbded75d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:05.971062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab56733-ec58-49f8-9a29-78f9cbded75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:05.991249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:06.045595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dab56733-ec58-49f8-9a29-78f9cbded75d None None] MFA Rules not processed for user `71a48b3b4fa94798b158488f08bae743`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:06.046782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dab56733-ec58-49f8-9a29-78f9cbded75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:06.121127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 669/1321] 10.222.0.33 () {58 vars in 1061 bytes} [Sat May 28 17:17:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:06.130124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305b8c2-cf37-4298-808d-263be52c9364 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:06.131700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b305b8c2-cf37-4298-808d-263be52c9364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:06.232921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305b8c2-cf37-4298-808d-263be52c9364 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:06.304152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763117b2-abc8-40bf-b7f7-1097ef58b3ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:06.304793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763117b2-abc8-40bf-b7f7-1097ef58b3ee service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:06.305078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763117b2-abc8-40bf-b7f7-1097ef58b3ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:06.305483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763117b2-abc8-40bf-b7f7-1097ef58b3ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:06.307143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-763117b2-abc8-40bf-b7f7-1097ef58b3ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:06.314673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:06.415253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763117b2-abc8-40bf-b7f7-1097ef58b3ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:06.416713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763117b2-abc8-40bf-b7f7-1097ef58b3ee service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:06.435741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 653/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:09.793309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8af6-a052-49b1-8db9-2d59fde465e3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:09.793864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8af6-a052-49b1-8db9-2d59fde465e3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:09.794008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8af6-a052-49b1-8db9-2d59fde465e3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:09.795502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 670/1323] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:09.800814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b177d-1da3-4c7e-ab09-b045cd94c9ae None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:09.801225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b177d-1da3-4c7e-ab09-b045cd94c9ae None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:09.801371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b177d-1da3-4c7e-ab09-b045cd94c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:09.866696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-696b177d-1da3-4c7e-ab09-b045cd94c9ae None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:09.868180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-696b177d-1da3-4c7e-ab09-b045cd94c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:09.925048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:09.966690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 654/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:10.002090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb8af6-a052-49b1-8db9-2d59fde465e3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:10.003663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-77bb8af6-a052-49b1-8db9-2d59fde465e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:10.108898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb8af6-a052-49b1-8db9-2d59fde465e3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:10.121447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:10.184652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83c41-f3ea-499c-bc8d-ae16d38bc052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:10.185302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83c41-f3ea-499c-bc8d-ae16d38bc052 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:10.185582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83c41-f3ea-499c-bc8d-ae16d38bc052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:10.185858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83c41-f3ea-499c-bc8d-ae16d38bc052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:10.187557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-80e83c41-f3ea-499c-bc8d-ae16d38bc052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:10.298391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e83c41-f3ea-499c-bc8d-ae16d38bc052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:10.299805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e83c41-f3ea-499c-bc8d-ae16d38bc052 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:10.318251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 671/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:12.616705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f3d1d-d12d-4bc1-8420-9231d5b794ea None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:12.617246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f3d1d-d12d-4bc1-8420-9231d5b794ea None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:12.617246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f3d1d-d12d-4bc1-8420-9231d5b794ea None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:12.619794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 655/1326] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:12.624881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fffb76-d702-4f5b-9d27-086273fb39c8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:12.625280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fffb76-d702-4f5b-9d27-086273fb39c8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:12.625429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fffb76-d702-4f5b-9d27-086273fb39c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:12.673035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:12.694664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-31fffb76-d702-4f5b-9d27-086273fb39c8 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:12.696389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-31fffb76-d702-4f5b-9d27-086273fb39c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:12.801930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 672/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:12.838199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f3d1d-d12d-4bc1-8420-9231d5b794ea None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:12.839802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5f6f3d1d-d12d-4bc1-8420-9231d5b794ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:12.941205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f3d1d-d12d-4bc1-8420-9231d5b794ea None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:13.012716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06776eb-b1d5-4acf-b1d5-1e0630b22d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:13.013370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06776eb-b1d5-4acf-b1d5-1e0630b22d22 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:13.013652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06776eb-b1d5-4acf-b1d5-1e0630b22d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:13.013929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06776eb-b1d5-4acf-b1d5-1e0630b22d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:13.015599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e06776eb-b1d5-4acf-b1d5-1e0630b22d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:13.032457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:13.129321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06776eb-b1d5-4acf-b1d5-1e0630b22d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:13.130957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06776eb-b1d5-4acf-b1d5-1e0630b22d22 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:13.152313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 656/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:15.450617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9ed9-c6ed-4ba6-a67e-e8e56d811b05 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:15.451184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9ed9-c6ed-4ba6-a67e-e8e56d811b05 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:15.451467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9ed9-c6ed-4ba6-a67e-e8e56d811b05 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:15.452876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 673/1329] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:15.458114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffa08d-23fb-48fa-9a1e-40e7317430ba None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:15.458536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffa08d-23fb-48fa-9a1e-40e7317430ba None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:15.458793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffa08d-23fb-48fa-9a1e-40e7317430ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:15.525615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8fffa08d-23fb-48fa-9a1e-40e7317430ba None None] MFA Rules not processed for user `61bb099d65764401bb3305bd6d0703a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:15.526839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8fffa08d-23fb-48fa-9a1e-40e7317430ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:15.594418 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:15.628301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 657/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:15.665799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9ed9-c6ed-4ba6-a67e-e8e56d811b05 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:15.667370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-14ba9ed9-c6ed-4ba6-a67e-e8e56d811b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:15.772982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9ed9-c6ed-4ba6-a67e-e8e56d811b05 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:15.793120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:15.847560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18c12f-4de2-4491-b0bb-4c0ef6f9d775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:15.848208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18c12f-4de2-4491-b0bb-4c0ef6f9d775 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:15.848491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18c12f-4de2-4491-b0bb-4c0ef6f9d775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:15.848768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18c12f-4de2-4491-b0bb-4c0ef6f9d775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:15.850438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2a18c12f-4de2-4491-b0bb-4c0ef6f9d775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:15.960520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18c12f-4de2-4491-b0bb-4c0ef6f9d775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:15.962129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18c12f-4de2-4491-b0bb-4c0ef6f9d775 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:15.980101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 674/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:18.457662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354fe94-111b-4246-8633-a8fd4831f6da None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:18.458221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354fe94-111b-4246-8633-a8fd4831f6da None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:18.458221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354fe94-111b-4246-8633-a8fd4831f6da None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:18.459645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 658/1332] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:18.465180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a597b39-db66-4c5b-8a7f-4ade17f536bb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:18.465596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a597b39-db66-4c5b-8a7f-4ade17f536bb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:18.465706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a597b39-db66-4c5b-8a7f-4ade17f536bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:18.532065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:18.534941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4a597b39-db66-4c5b-8a7f-4ade17f536bb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:18.536732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4a597b39-db66-4c5b-8a7f-4ade17f536bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:18.635817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 675/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:18.671025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354fe94-111b-4246-8633-a8fd4831f6da None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:18.672604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8354fe94-111b-4246-8633-a8fd4831f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:18.776905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354fe94-111b-4246-8633-a8fd4831f6da None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:18.848574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92aa49-a5ca-4600-9be3-ed723c65e56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:18.849221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92aa49-a5ca-4600-9be3-ed723c65e56e service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:18.849500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92aa49-a5ca-4600-9be3-ed723c65e56e service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:18.849803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92aa49-a5ca-4600-9be3-ed723c65e56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:18.851504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3a92aa49-a5ca-4600-9be3-ed723c65e56e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:18.881625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:18.971836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92aa49-a5ca-4600-9be3-ed723c65e56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:18.973230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92aa49-a5ca-4600-9be3-ed723c65e56e service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:18.994366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 659/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:21.286837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9d1a5-725f-40a9-a91b-449e333a6da7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:21.287320 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9d1a5-725f-40a9-a91b-449e333a6da7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:21.287556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9d1a5-725f-40a9-a91b-449e333a6da7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:21.288807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 676/1335] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:21.294658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc191-b01e-4e28-9bbe-659d46cfa1ab None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:21.295065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc191-b01e-4e28-9bbe-659d46cfa1ab None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:21.295222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc191-b01e-4e28-9bbe-659d46cfa1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:21.359555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4d1fc191-b01e-4e28-9bbe-659d46cfa1ab None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:21.360970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4d1fc191-b01e-4e28-9bbe-659d46cfa1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:21.433926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:21.456617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 660/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:21.497674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9d1a5-725f-40a9-a91b-449e333a6da7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:21.499246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8db9d1a5-725f-40a9-a91b-449e333a6da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:21.600696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9d1a5-725f-40a9-a91b-449e333a6da7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:21.629558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:21.674570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2738791-bd55-4706-806c-0c7eec135cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:21.675219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2738791-bd55-4706-806c-0c7eec135cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:21.675530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2738791-bd55-4706-806c-0c7eec135cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:21.675813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2738791-bd55-4706-806c-0c7eec135cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:21.677461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f2738791-bd55-4706-806c-0c7eec135cb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:21.786896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2738791-bd55-4706-806c-0c7eec135cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:21.788360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2738791-bd55-4706-806c-0c7eec135cb9 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:21.806975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 677/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:24.431234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c23bfc-1edb-4fa9-91fb-a84ca2508ffd None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:24.431886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c23bfc-1edb-4fa9-91fb-a84ca2508ffd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:24.431886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c23bfc-1edb-4fa9-91fb-a84ca2508ffd None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:24.433234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 661/1338] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:24.438418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa2b9-fbab-48ac-8566-c9d48573d9f3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:24.438873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa2b9-fbab-48ac-8566-c9d48573d9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:24.438985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa2b9-fbab-48ac-8566-c9d48573d9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:24.509047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9dafa2b9-fbab-48ac-8566-c9d48573d9f3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:24.510263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9dafa2b9-fbab-48ac-8566-c9d48573d9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:24.518393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:24.606203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 678/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:24.642086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c23bfc-1edb-4fa9-91fb-a84ca2508ffd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:24.643649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42c23bfc-1edb-4fa9-91fb-a84ca2508ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:24.745911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c23bfc-1edb-4fa9-91fb-a84ca2508ffd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:24.817200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b1009-1f3c-49de-bd2a-2422f7a031c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:24.817840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b1009-1f3c-49de-bd2a-2422f7a031c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:24.818118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b1009-1f3c-49de-bd2a-2422f7a031c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:24.818388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b1009-1f3c-49de-bd2a-2422f7a031c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:24.820083 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e83b1009-1f3c-49de-bd2a-2422f7a031c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:24.855670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:24.927307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b1009-1f3c-49de-bd2a-2422f7a031c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:24.928751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b1009-1f3c-49de-bd2a-2422f7a031c7 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:24.946986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 662/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:29.288709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d54940-7e04-4f0f-978f-ca85b6cb6eaf None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:29.289252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d54940-7e04-4f0f-978f-ca85b6cb6eaf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:29.289252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d54940-7e04-4f0f-978f-ca85b6cb6eaf None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:29.290544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 679/1341] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:29.295712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b67376-0881-4d0e-8520-8774f80d1a1c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:29.296138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b67376-0881-4d0e-8520-8774f80d1a1c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:29.296272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b67376-0881-4d0e-8520-8774f80d1a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:29.373081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a4b67376-0881-4d0e-8520-8774f80d1a1c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:29.374286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a4b67376-0881-4d0e-8520-8774f80d1a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:29.470085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:29.487647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 663/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:29.528149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d54940-7e04-4f0f-978f-ca85b6cb6eaf None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:29.530431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f5d54940-7e04-4f0f-978f-ca85b6cb6eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:29.627360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d54940-7e04-4f0f-978f-ca85b6cb6eaf None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:29.630856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c9d9e-24e3-4c11-a5e6-81f5435175f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:29.631528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9d9e-24e3-4c11-a5e6-81f5435175f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:29.631808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9d9e-24e3-4c11-a5e6-81f5435175f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:29.632081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9d9e-24e3-4c11-a5e6-81f5435175f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:29.642818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c9d9e-24e3-4c11-a5e6-81f5435175f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:29.644303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c9d9e-24e3-4c11-a5e6-81f5435175f8 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:29.654294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 680/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:32.463760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbbf30-ab3d-4390-a861-244d24f4fe56 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:32.464310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbbf30-ab3d-4390-a861-244d24f4fe56 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:32.464310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbbf30-ab3d-4390-a861-244d24f4fe56 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:32.465685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 664/1344] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:32.470751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda47e62-896c-460b-a3d1-a1ed19a9d903 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:32.471171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda47e62-896c-460b-a3d1-a1ed19a9d903 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:32.471381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda47e62-896c-460b-a3d1-a1ed19a9d903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:32.504015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cda47e62-896c-460b-a3d1-a1ed19a9d903 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:32.505309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cda47e62-896c-460b-a3d1-a1ed19a9d903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:32.535650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:32.585026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 681/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:32.621972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbbf30-ab3d-4390-a861-244d24f4fe56 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:32.633200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbbf30-ab3d-4390-a861-244d24f4fe56 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:32.636840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e5022-255f-4694-b70d-22b3dfb6740c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:32.637472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e5022-255f-4694-b70d-22b3dfb6740c service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:32.637752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e5022-255f-4694-b70d-22b3dfb6740c service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:32.638025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e5022-255f-4694-b70d-22b3dfb6740c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:32.646989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e5022-255f-4694-b70d-22b3dfb6740c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:32.648616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e5022-255f-4694-b70d-22b3dfb6740c service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:32.658026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 665/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:36.799637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6f56f-af67-4b0a-820c-c1cd5c5c9bc1 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:36.800161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6f56f-af67-4b0a-820c-c1cd5c5c9bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:36.800273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6f56f-af67-4b0a-820c-c1cd5c5c9bc1 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:36.801568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 682/1347] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:36.807563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfeaee-56c2-4df5-9f93-bdc4555a6f1a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:36.808008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfeaee-56c2-4df5-9f93-bdc4555a6f1a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:36.808155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfeaee-56c2-4df5-9f93-bdc4555a6f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:36.826016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-31cfeaee-56c2-4df5-9f93-bdc4555a6f1a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:36.827221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-31cfeaee-56c2-4df5-9f93-bdc4555a6f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:36.835584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 666/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:36.875393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6f56f-af67-4b0a-820c-c1cd5c5c9bc1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:36.886070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6f56f-af67-4b0a-820c-c1cd5c5c9bc1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:36.889561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafbf8b-fbce-4df2-a4ca-1d74390ccbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:36.890195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafbf8b-fbce-4df2-a4ca-1d74390ccbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:36.890476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafbf8b-fbce-4df2-a4ca-1d74390ccbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:36.890751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafbf8b-fbce-4df2-a4ca-1d74390ccbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:36.900207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafbf8b-fbce-4df2-a4ca-1d74390ccbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:36.901739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafbf8b-fbce-4df2-a4ca-1d74390ccbf8 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:36.912919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 683/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:42.302493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6901e6-a572-48a3-a484-5cc2700bd943 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:42.303620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6901e6-a572-48a3-a484-5cc2700bd943 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:42.304103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6901e6-a572-48a3-a484-5cc2700bd943 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:42.305729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 667/1350] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:17:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:42.315138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f4ef5-7a8f-4145-abef-936ff06315fa None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:42.315579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f4ef5-7a8f-4145-abef-936ff06315fa None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:42.315730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f4ef5-7a8f-4145-abef-936ff06315fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:42.421870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7e3f4ef5-7a8f-4145-abef-936ff06315fa None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:42.423066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7e3f4ef5-7a8f-4145-abef-936ff06315fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:42.514244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 684/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:17:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:46.684707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e25b-752c-4606-bb9b-0c4acbaa046e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:46.685490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e25b-752c-4606-bb9b-0c4acbaa046e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:46.685490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e25b-752c-4606-bb9b-0c4acbaa046e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:46.687076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 668/1352] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:46.694381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ac00-5f02-477f-85d2-778563316c3a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:46.696000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ac00-5f02-477f-85d2-778563316c3a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:46.696925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ac00-5f02-477f-85d2-778563316c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:46.719303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ec75ac00-5f02-477f-85d2-778563316c3a None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:46.720838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ec75ac00-5f02-477f-85d2-778563316c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:46.731461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 685/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:17:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:17:46.820843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=13 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:48.577471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461cf4d-a939-4654-a46e-88aeeb2ed048 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:48.578169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461cf4d-a939-4654-a46e-88aeeb2ed048 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:48.578169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461cf4d-a939-4654-a46e-88aeeb2ed048 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:48.579433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 669/1354] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:48.584718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b0644-aec2-4420-ac86-dc4f1ed3354b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:48.585369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b0644-aec2-4420-ac86-dc4f1ed3354b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:48.585369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b0644-aec2-4420-ac86-dc4f1ed3354b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:48.605387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d18b0644-aec2-4420-ac86-dc4f1ed3354b None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:48.606662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d18b0644-aec2-4420-ac86-dc4f1ed3354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:48.616091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 686/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:17:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:52.361800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74aca6-7c45-4b7c-b3d8-daec6c34e201 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:52.363088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74aca6-7c45-4b7c-b3d8-daec6c34e201 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:52.363645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74aca6-7c45-4b7c-b3d8-daec6c34e201 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:52.366044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 670/1356] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:52.372045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d10f-892a-4b2f-9752-6a482d39d5e6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:52.372917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d10f-892a-4b2f-9752-6a482d39d5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:52.373434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d10f-892a-4b2f-9752-6a482d39d5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:52.392460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-24a2d10f-892a-4b2f-9752-6a482d39d5e6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:52.393739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-24a2d10f-892a-4b2f-9752-6a482d39d5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:52.402543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 687/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:52.461114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a454248-f4db-4fc1-a01f-55e08d7925f1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:52.461773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a454248-f4db-4fc1-a01f-55e08d7925f1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:52.462237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a454248-f4db-4fc1-a01f-55e08d7925f1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:52.464593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 671/1358] 10.222.0.33 () {54 vars in 961 bytes} [Sat May 28 17:17:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:52.473673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa61d-8ff9-43f2-96bb-e84c82c21b59 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:52.474366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa61d-8ff9-43f2-96bb-e84c82c21b59 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:52.476022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa61d-8ff9-43f2-96bb-e84c82c21b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:52.524059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:17:52.531645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-eddfa61d-8ff9-43f2-96bb-e84c82c21b59 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:52.533639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eddfa61d-8ff9-43f2-96bb-e84c82c21b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:52.621557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 688/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 17:17:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:17:52.630056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a454248-f4db-4fc1-a01f-55e08d7925f1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:52.645132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a454248-f4db-4fc1-a01f-55e08d7925f1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:52.650712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccff885-9d53-40e3-94e3-d2c724ab38e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:52.651374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff885-9d53-40e3-94e3-d2c724ab38e2 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:52.651659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff885-9d53-40e3-94e3-d2c724ab38e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:52.651939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff885-9d53-40e3-94e3-d2c724ab38e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:52.660165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccff885-9d53-40e3-94e3-d2c724ab38e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:52.661765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccff885-9d53-40e3-94e3-d2c724ab38e2 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:52.671420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 672/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:55.141219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b097aa-fa6b-44fc-94b2-0d4b658cc146 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:55.141790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b097aa-fa6b-44fc-94b2-0d4b658cc146 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:55.141790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b097aa-fa6b-44fc-94b2-0d4b658cc146 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:55.143276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 689/1361] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:55.149336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9e16-3543-4127-a4c9-679e598d715d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:55.149812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9e16-3543-4127-a4c9-679e598d715d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:55.149904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9e16-3543-4127-a4c9-679e598d715d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:55.169548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1f8c9e16-3543-4127-a4c9-679e598d715d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:55.170737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f8c9e16-3543-4127-a4c9-679e598d715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:55.179115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 673/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:55.223677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b097aa-fa6b-44fc-94b2-0d4b658cc146 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:55.241894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b097aa-fa6b-44fc-94b2-0d4b658cc146 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:55.248050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2080da4-ff78-4357-aabc-74c9b1625c77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:55.248708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2080da4-ff78-4357-aabc-74c9b1625c77 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:55.248997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2080da4-ff78-4357-aabc-74c9b1625c77 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:55.249274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2080da4-ff78-4357-aabc-74c9b1625c77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:55.259629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2080da4-ff78-4357-aabc-74c9b1625c77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:55.261417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2080da4-ff78-4357-aabc-74c9b1625c77 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:55.271475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 690/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:17:57.809112 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73304e-7dca-47ad-92ab-9766b881bcc5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:57.809664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73304e-7dca-47ad-92ab-9766b881bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:57.809664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73304e-7dca-47ad-92ab-9766b881bcc5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:57.811109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 674/1364] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:57.816249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081b709-07be-4112-ad1c-107472c02f52 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:57.816664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081b709-07be-4112-ad1c-107472c02f52 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:57.816809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081b709-07be-4112-ad1c-107472c02f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:57.836007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7081b709-07be-4112-ad1c-107472c02f52 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:57.837200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7081b709-07be-4112-ad1c-107472c02f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:57.844956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 691/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:57.890263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbec4a-4013-4efc-9512-ae06b9948f4c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:57.890624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbec4a-4013-4efc-9512-ae06b9948f4c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:57.890972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbec4a-4013-4efc-9512-ae06b9948f4c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:57.892455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 675/1366] 10.222.0.33 () {54 vars in 961 bytes} [Sat May 28 17:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:17:57.897899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecff897-8648-4103-9ace-bab4c3fae819 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:57.898260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecff897-8648-4103-9ace-bab4c3fae819 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:57.898695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecff897-8648-4103-9ace-bab4c3fae819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:57.917648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2ecff897-8648-4103-9ace-bab4c3fae819 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:17:57.918851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2ecff897-8648-4103-9ace-bab4c3fae819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:17:57.929700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 692/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 17:17:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:17:57.944115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbec4a-4013-4efc-9512-ae06b9948f4c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:17:57.971297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbec4a-4013-4efc-9512-ae06b9948f4c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:17:57.978282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f780eb-d8d1-4c7c-ac0d-520f32165927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:17:57.979023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f780eb-d8d1-4c7c-ac0d-520f32165927 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:17:57.979430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f780eb-d8d1-4c7c-ac0d-520f32165927 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:17:57.979679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f780eb-d8d1-4c7c-ac0d-520f32165927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:17:57.993447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f780eb-d8d1-4c7c-ac0d-520f32165927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:17:57.995890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f780eb-d8d1-4c7c-ac0d-520f32165927 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:17:58.007525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 676/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:00.374908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586b6f4-fbd1-40ee-a731-e250f4b03819 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:00.375941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586b6f4-fbd1-40ee-a731-e250f4b03819 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:00.376457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586b6f4-fbd1-40ee-a731-e250f4b03819 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:00.378446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 693/1369] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:00.384456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776dd3c-b2ab-4d7d-8444-50bcfdfbc79e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:00.384912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776dd3c-b2ab-4d7d-8444-50bcfdfbc79e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:00.385008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776dd3c-b2ab-4d7d-8444-50bcfdfbc79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:00.403426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4776dd3c-b2ab-4d7d-8444-50bcfdfbc79e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:00.404619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4776dd3c-b2ab-4d7d-8444-50bcfdfbc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:00.412987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 677/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:00.458194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586b6f4-fbd1-40ee-a731-e250f4b03819 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:00.476555 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586b6f4-fbd1-40ee-a731-e250f4b03819 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:00.483812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85446094-da71-447d-be1c-48290a9b8c6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:00.484760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85446094-da71-447d-be1c-48290a9b8c6d service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:00.485251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85446094-da71-447d-be1c-48290a9b8c6d service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:00.485734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85446094-da71-447d-be1c-48290a9b8c6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:00.497664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85446094-da71-447d-be1c-48290a9b8c6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:00.499540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85446094-da71-447d-be1c-48290a9b8c6d service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:00.511080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 694/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:00.746354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eecd1-932a-4645-ac8a-279803559317 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:00.746642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eecd1-932a-4645-ac8a-279803559317 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:00.747111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eecd1-932a-4645-ac8a-279803559317 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:00.748632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 678/1372] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:00.755002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15886a05-c61b-419f-86e8-5d8a4c737f5d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:00.756105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15886a05-c61b-419f-86e8-5d8a4c737f5d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:00.756828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15886a05-c61b-419f-86e8-5d8a4c737f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:00.778851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-15886a05-c61b-419f-86e8-5d8a4c737f5d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:00.780138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-15886a05-c61b-419f-86e8-5d8a4c737f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:00.788424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 695/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:01.094243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8fc6-39b8-494b-a1f6-161f1f74d952 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:01.094243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8fc6-39b8-494b-a1f6-161f1f74d952 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:01.094535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8fc6-39b8-494b-a1f6-161f1f74d952 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:01.096090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 679/1374] 10.222.0.33 () {54 vars in 963 bytes} [Sat May 28 17:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:01.102138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7afaf-9a06-4e80-bd80-ef608f804e72 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:01.102502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7afaf-9a06-4e80-bd80-ef608f804e72 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:01.102848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7afaf-9a06-4e80-bd80-ef608f804e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:01.159405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e8a7afaf-9a06-4e80-bd80-ef608f804e72 None None] MFA Rules not processed for user `57af250e151241f4afc9b38d51a5fbdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:01.160606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e8a7afaf-9a06-4e80-bd80-ef608f804e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:01.223692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 696/1375] 10.222.0.33 () {58 vars in 1060 bytes} [Sat May 28 17:18:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:01.231853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f8fc6-39b8-494b-a1f6-161f1f74d952 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:01.243859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f8fc6-39b8-494b-a1f6-161f1f74d952 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:01.247421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f04fc3-c588-457f-8d6d-3d44138d4f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:01.248116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f04fc3-c588-457f-8d6d-3d44138d4f73 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:01.248405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f04fc3-c588-457f-8d6d-3d44138d4f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:01.248688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f04fc3-c588-457f-8d6d-3d44138d4f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:01.258477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f04fc3-c588-457f-8d6d-3d44138d4f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:01.259958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f04fc3-c588-457f-8d6d-3d44138d4f73 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:01.270151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 680/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:02.779230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cafd1-c855-4dc7-a65f-0c8e8b5da88c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:02.779822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cafd1-c855-4dc7-a65f-0c8e8b5da88c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:02.779822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cafd1-c855-4dc7-a65f-0c8e8b5da88c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:02.781228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 697/1377] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:02.787382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec86ca0-8fbc-4db8-89d1-16e1eb4bd149 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:02.787959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec86ca0-8fbc-4db8-89d1-16e1eb4bd149 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:02.788052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec86ca0-8fbc-4db8-89d1-16e1eb4bd149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:02.807118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cec86ca0-8fbc-4db8-89d1-16e1eb4bd149 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:02.808387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cec86ca0-8fbc-4db8-89d1-16e1eb4bd149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:02.816445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 681/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:02.862092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cafd1-c855-4dc7-a65f-0c8e8b5da88c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:02.880909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cafd1-c855-4dc7-a65f-0c8e8b5da88c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:02.887100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398e770-3177-4777-b364-a48ff2718fa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:02.887803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e770-3177-4777-b364-a48ff2718fa6 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:02.888113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e770-3177-4777-b364-a48ff2718fa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:02.888389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e770-3177-4777-b364-a48ff2718fa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:02.898580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398e770-3177-4777-b364-a48ff2718fa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:02.900258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398e770-3177-4777-b364-a48ff2718fa6 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:02.905186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:18:02.913365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 698/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:05.178236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c3dc8-d3a7-474a-b45f-a9b28f25beee None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:05.178785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c3dc8-d3a7-474a-b45f-a9b28f25beee None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:05.178785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c3dc8-d3a7-474a-b45f-a9b28f25beee None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:05.180073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 682/1380] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:05.185817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b397ed9-93e3-4f7e-af42-cc851d1bcc81 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:05.186270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b397ed9-93e3-4f7e-af42-cc851d1bcc81 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:05.186384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b397ed9-93e3-4f7e-af42-cc851d1bcc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:05.210211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0b397ed9-93e3-4f7e-af42-cc851d1bcc81 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:05.211718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0b397ed9-93e3-4f7e-af42-cc851d1bcc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:05.219867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 699/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:05.264966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c3dc8-d3a7-474a-b45f-a9b28f25beee None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:05.282494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c3dc8-d3a7-474a-b45f-a9b28f25beee None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:05.288560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecbb75-f262-4c5d-b693-ebeee7aaecd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:05.289207 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecbb75-f262-4c5d-b693-ebeee7aaecd7 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:05.289485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecbb75-f262-4c5d-b693-ebeee7aaecd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:05.289760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecbb75-f262-4c5d-b693-ebeee7aaecd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:05.298309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecbb75-f262-4c5d-b693-ebeee7aaecd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:05.299904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecbb75-f262-4c5d-b693-ebeee7aaecd7 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:05.309942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 683/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:07.618334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea445542-71be-49bf-843f-6a01470c0c5f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:07.618864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea445542-71be-49bf-843f-6a01470c0c5f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:07.618945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea445542-71be-49bf-843f-6a01470c0c5f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:07.620453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 700/1383] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:07.625674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be75f3-9519-4cec-afeb-3155b7968f3d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:07.626077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be75f3-9519-4cec-afeb-3155b7968f3d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:07.626224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be75f3-9519-4cec-afeb-3155b7968f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:07.645100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-28be75f3-9519-4cec-afeb-3155b7968f3d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:07.646982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-28be75f3-9519-4cec-afeb-3155b7968f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:07.655280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 684/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:07.697466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea445542-71be-49bf-843f-6a01470c0c5f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:07.717313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea445542-71be-49bf-843f-6a01470c0c5f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:07.722965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712096fc-e8ce-4dd0-8c94-433395236421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:07.723770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712096fc-e8ce-4dd0-8c94-433395236421 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:07.724221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712096fc-e8ce-4dd0-8c94-433395236421 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:07.724573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712096fc-e8ce-4dd0-8c94-433395236421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:07.733990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712096fc-e8ce-4dd0-8c94-433395236421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:07.735780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712096fc-e8ce-4dd0-8c94-433395236421 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:07.746980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 701/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:09.990556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49222677-1084-4c22-a60d-611fd8a1bfb5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:09.991322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49222677-1084-4c22-a60d-611fd8a1bfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:09.991322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49222677-1084-4c22-a60d-611fd8a1bfb5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:09.992666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 685/1386] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:09.998484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506af1b1-cdd1-4a1b-b2e6-4756b5aa3b65 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:09.999490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506af1b1-cdd1-4a1b-b2e6-4756b5aa3b65 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:09.999885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506af1b1-cdd1-4a1b-b2e6-4756b5aa3b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:10.021238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-506af1b1-cdd1-4a1b-b2e6-4756b5aa3b65 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:10.022929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-506af1b1-cdd1-4a1b-b2e6-4756b5aa3b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:10.033020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 702/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:10.078680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49222677-1084-4c22-a60d-611fd8a1bfb5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:10.093818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49222677-1084-4c22-a60d-611fd8a1bfb5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:10.100000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b1757-cd31-424b-a6b2-b2e23bfbe2d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:10.100639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1757-cd31-424b-a6b2-b2e23bfbe2d6 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:10.100921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1757-cd31-424b-a6b2-b2e23bfbe2d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:10.101208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1757-cd31-424b-a6b2-b2e23bfbe2d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:10.109772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b1757-cd31-424b-a6b2-b2e23bfbe2d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:10.111374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b1757-cd31-424b-a6b2-b2e23bfbe2d6 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:10.121561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 686/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:12.362665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf321-5967-4c39-bfcc-2c197969f538 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:12.363192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf321-5967-4c39-bfcc-2c197969f538 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:12.363408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf321-5967-4c39-bfcc-2c197969f538 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:12.366234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 703/1389] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:12.373263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4400de-7647-48a4-a9ed-b7b8d7dfe9cc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:12.373794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4400de-7647-48a4-a9ed-b7b8d7dfe9cc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:12.374137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4400de-7647-48a4-a9ed-b7b8d7dfe9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:12.398938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ca4400de-7647-48a4-a9ed-b7b8d7dfe9cc None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:12.400606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ca4400de-7647-48a4-a9ed-b7b8d7dfe9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:12.410869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 687/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:12.462557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abf321-5967-4c39-bfcc-2c197969f538 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:12.482192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abf321-5967-4c39-bfcc-2c197969f538 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:12.489884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993de0b-583b-4d9d-bc84-9f2521049c1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:12.490630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993de0b-583b-4d9d-bc84-9f2521049c1b service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:12.490914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993de0b-583b-4d9d-bc84-9f2521049c1b service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:12.491510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993de0b-583b-4d9d-bc84-9f2521049c1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:12.500311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993de0b-583b-4d9d-bc84-9f2521049c1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:12.501909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993de0b-583b-4d9d-bc84-9f2521049c1b service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:12.511542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 704/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:15.180197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac7605-0c76-40da-a9fd-2d3474305a44 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:15.180782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac7605-0c76-40da-a9fd-2d3474305a44 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:15.180782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac7605-0c76-40da-a9fd-2d3474305a44 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:15.182123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 688/1392] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:15.187528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5f38e-a821-4709-925c-a0bcb7bee8ab None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:15.187973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5f38e-a821-4709-925c-a0bcb7bee8ab None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:15.188074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5f38e-a821-4709-925c-a0bcb7bee8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:15.205744 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a3f5f38e-a821-4709-925c-a0bcb7bee8ab None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:15.206946 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a3f5f38e-a821-4709-925c-a0bcb7bee8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:15.215315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 705/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:15.260449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac7605-0c76-40da-a9fd-2d3474305a44 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:15.275248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac7605-0c76-40da-a9fd-2d3474305a44 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:15.281061 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5fbed-8f9e-4dbe-8617-96ee9f100c52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:15.281704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5fbed-8f9e-4dbe-8617-96ee9f100c52 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:15.281987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5fbed-8f9e-4dbe-8617-96ee9f100c52 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:15.282266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5fbed-8f9e-4dbe-8617-96ee9f100c52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:15.291666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5fbed-8f9e-4dbe-8617-96ee9f100c52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:15.293254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5fbed-8f9e-4dbe-8617-96ee9f100c52 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:15.303285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 689/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:17.003392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878aaf-9afb-4926-ae8c-4763feaf0a3f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.003929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878aaf-9afb-4926-ae8c-4763feaf0a3f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.003929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878aaf-9afb-4926-ae8c-4763feaf0a3f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.005331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 706/1395] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:17.010674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cb442-4048-4319-8e7d-eb29e3213747 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.011118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cb442-4048-4319-8e7d-eb29e3213747 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.011229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cb442-4048-4319-8e7d-eb29e3213747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.017175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=52 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:18:17.030130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-107cb442-4048-4319-8e7d-eb29e3213747 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:17.031363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-107cb442-4048-4319-8e7d-eb29e3213747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:17.039869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 690/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:17.095686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747754d-8968-4cf2-94dc-2330b7ba1a79 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.095979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747754d-8968-4cf2-94dc-2330b7ba1a79 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.096258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747754d-8968-4cf2-94dc-2330b7ba1a79 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.097637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 707/1397] 10.222.0.33 () {54 vars in 962 bytes} [Sat May 28 17:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:17.103066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af65bbb-3ebc-4813-bb9e-d6ce298ba463 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.103382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af65bbb-3ebc-4813-bb9e-d6ce298ba463 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.103668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af65bbb-3ebc-4813-bb9e-d6ce298ba463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.152309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7af65bbb-3ebc-4813-bb9e-d6ce298ba463 None None] MFA Rules not processed for user `58b8dd344e8e43ec9cec222737f98ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:17.153532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7af65bbb-3ebc-4813-bb9e-d6ce298ba463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:17.214438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 691/1398] 10.222.0.33 () {58 vars in 1059 bytes} [Sat May 28 17:18:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:17.221972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747754d-8968-4cf2-94dc-2330b7ba1a79 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:17.233260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747754d-8968-4cf2-94dc-2330b7ba1a79 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:17.236772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a85031-bc8b-4c4f-aa8a-f409d385ef22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:17.237406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a85031-bc8b-4c4f-aa8a-f409d385ef22 service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.237698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a85031-bc8b-4c4f-aa8a-f409d385ef22 service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.237972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a85031-bc8b-4c4f-aa8a-f409d385ef22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.248089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a85031-bc8b-4c4f-aa8a-f409d385ef22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:17.249564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a85031-bc8b-4c4f-aa8a-f409d385ef22 service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:17.259945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 708/1399] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:17.637156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1b1d-3c9d-44fc-b865-2063be5ff90b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.637481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1b1d-3c9d-44fc-b865-2063be5ff90b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.637791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1b1d-3c9d-44fc-b865-2063be5ff90b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.639170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 692/1400] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:17.645698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91324fbb-114f-4848-b2ac-a8d0873ab116 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.646165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91324fbb-114f-4848-b2ac-a8d0873ab116 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.646615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91324fbb-114f-4848-b2ac-a8d0873ab116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.664855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-91324fbb-114f-4848-b2ac-a8d0873ab116 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:17.666054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-91324fbb-114f-4848-b2ac-a8d0873ab116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:17.673774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 709/1401] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:18:17.720231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada1b1d-3c9d-44fc-b865-2063be5ff90b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:17.735148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada1b1d-3c9d-44fc-b865-2063be5ff90b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:17.742100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea75e04-4b02-4bc5-9f16-9ac7198b1cd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:17.743116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea75e04-4b02-4bc5-9f16-9ac7198b1cd7 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.743458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea75e04-4b02-4bc5-9f16-9ac7198b1cd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.743771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea75e04-4b02-4bc5-9f16-9ac7198b1cd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.752918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea75e04-4b02-4bc5-9f16-9ac7198b1cd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:17.755029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea75e04-4b02-4bc5-9f16-9ac7198b1cd7 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:17.764953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 693/1402] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:17.794226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0131aa-f8c2-4795-9655-bf774766c4bb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.794731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0131aa-f8c2-4795-9655-bf774766c4bb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.795076 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0131aa-f8c2-4795-9655-bf774766c4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.846518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0d0131aa-f8c2-4795-9655-bf774766c4bb None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:17.848071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0d0131aa-f8c2-4795-9655-bf774766c4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:17.923132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 710/1403] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:18:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:17.970572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b6cd-c5b8-4291-a2cd-55f2966f96b5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.970858 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b6cd-c5b8-4291-a2cd-55f2966f96b5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.971136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b6cd-c5b8-4291-a2cd-55f2966f96b5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:17.972547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 694/1404] 10.222.0.33 () {54 vars in 942 bytes} [Sat May 28 17:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:17.979756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218817c7-e09b-4349-a617-819a89f88484 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:17.980043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218817c7-e09b-4349-a617-819a89f88484 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:17.980314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218817c7-e09b-4349-a617-819a89f88484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:18.038226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-218817c7-e09b-4349-a617-819a89f88484 None None] MFA Rules not processed for user `c4224f58ecbb423dae16d5b9ae09d5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:18.039353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-218817c7-e09b-4349-a617-819a89f88484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:18.072970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:18:18.111241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 711/1405] 10.222.0.33 () {58 vars in 1039 bytes} [Sat May 28 17:18:17 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:18.121358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104b6cd-c5b8-4291-a2cd-55f2966f96b5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:18.135520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104b6cd-c5b8-4291-a2cd-55f2966f96b5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:18.140491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc1ad7-c9fa-4558-8bb4-d83c4daec455 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:18.141203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1ad7-c9fa-4558-8bb4-d83c4daec455 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:18.142085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1ad7-c9fa-4558-8bb4-d83c4daec455 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:18.142362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1ad7-c9fa-4558-8bb4-d83c4daec455 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:18.150789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc1ad7-c9fa-4558-8bb4-d83c4daec455 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:18.152394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc1ad7-c9fa-4558-8bb4-d83c4daec455 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:18.160253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 695/1406] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:18:20.092158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af96ad-c1f2-436d-a205-4d4135d8edac None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:20.092993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af96ad-c1f2-436d-a205-4d4135d8edac None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:20.092993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af96ad-c1f2-436d-a205-4d4135d8edac None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:20.094100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 712/1407] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:20.099833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bd9d0-f606-472d-8f31-1219e12fed91 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:20.100533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bd9d0-f606-472d-8f31-1219e12fed91 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:20.100533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bd9d0-f606-472d-8f31-1219e12fed91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:20.123994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a32bd9d0-f606-472d-8f31-1219e12fed91 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:20.125650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a32bd9d0-f606-472d-8f31-1219e12fed91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:20.135933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 696/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:20.180507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af96ad-c1f2-436d-a205-4d4135d8edac None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:20.196123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af96ad-c1f2-436d-a205-4d4135d8edac None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:20.201767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55536490-1ae7-4de5-94d7-7e3ffd9ed6a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:20.202421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536490-1ae7-4de5-94d7-7e3ffd9ed6a8 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:20.202705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536490-1ae7-4de5-94d7-7e3ffd9ed6a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:20.202981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536490-1ae7-4de5-94d7-7e3ffd9ed6a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:20.212655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55536490-1ae7-4de5-94d7-7e3ffd9ed6a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:20.214248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55536490-1ae7-4de5-94d7-7e3ffd9ed6a8 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:20.235067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 713/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:30.118006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=55 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:18:30.228302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=21 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:18:41.511381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8585383-7c5b-426f-84d1-5da9c6f61a62 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:41.512051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8585383-7c5b-426f-84d1-5da9c6f61a62 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:41.512051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8585383-7c5b-426f-84d1-5da9c6f61a62 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:41.513325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 697/1410] 10.222.0.33 () {54 vars in 918 bytes} [Sat May 28 17:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:41.518649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511c78-ea9b-4ae6-871c-c8856b0a8981 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:41.519684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511c78-ea9b-4ae6-871c-c8856b0a8981 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:41.519843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511c78-ea9b-4ae6-871c-c8856b0a8981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:41.538359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6b511c78-ea9b-4ae6-871c-c8856b0a8981 None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:41.539575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6b511c78-ea9b-4ae6-871c-c8856b0a8981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:41.547901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 714/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Sat May 28 17:18:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:41.561536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d87c6-15db-4a66-b3fb-24f01e084f7a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:41.561827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d87c6-15db-4a66-b3fb-24f01e084f7a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:41.562109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d87c6-15db-4a66-b3fb-24f01e084f7a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:41.563531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 698/1412] 10.222.0.33 () {54 vars in 918 bytes} [Sat May 28 17:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:41.567918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a842e7-d3a0-4108-baef-1027c222f6f7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:41.568208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a842e7-d3a0-4108-baef-1027c222f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:41.568492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a842e7-d3a0-4108-baef-1027c222f6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:41.586341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-49a842e7-d3a0-4108-baef-1027c222f6f7 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:41.587554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-49a842e7-d3a0-4108-baef-1027c222f6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:41.596614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 715/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Sat May 28 17:18:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:44.293953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83519b-20bf-4b20-bfcb-4be8ef8aa631 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:44.294509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83519b-20bf-4b20-bfcb-4be8ef8aa631 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:44.294509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83519b-20bf-4b20-bfcb-4be8ef8aa631 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:44.295823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:44.301619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084a1af-e2c3-4df9-b0ff-cddb2ea9e0ea None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:44.302010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084a1af-e2c3-4df9-b0ff-cddb2ea9e0ea None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:44.302176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084a1af-e2c3-4df9-b0ff-cddb2ea9e0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:44.319655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6084a1af-e2c3-4df9-b0ff-cddb2ea9e0ea None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:44.320849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6084a1af-e2c3-4df9-b0ff-cddb2ea9e0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:44.328544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:44.371494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83519b-20bf-4b20-bfcb-4be8ef8aa631 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:44.385906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83519b-20bf-4b20-bfcb-4be8ef8aa631 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:44.391186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300a4e6-cbf1-4a1a-a577-8c22028be6d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:44.391866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a4e6-cbf1-4a1a-a577-8c22028be6d7 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:44.392151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a4e6-cbf1-4a1a-a577-8c22028be6d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:44.392429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a4e6-cbf1-4a1a-a577-8c22028be6d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:44.401334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300a4e6-cbf1-4a1a-a577-8c22028be6d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:44.402912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300a4e6-cbf1-4a1a-a577-8c22028be6d7 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:44.412843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 700/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:45.388494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c26ff-4fdd-4f81-b8b4-b387d7e703f7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:45.389075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c26ff-4fdd-4f81-b8b4-b387d7e703f7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:45.389075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c26ff-4fdd-4f81-b8b4-b387d7e703f7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:45.390410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:45.395960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aec1b3-ec28-48e1-9c23-7988e0aa816a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:45.396397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aec1b3-ec28-48e1-9c23-7988e0aa816a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:45.396593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aec1b3-ec28-48e1-9c23-7988e0aa816a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:45.417265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-76aec1b3-ec28-48e1-9c23-7988e0aa816a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:45.418533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-76aec1b3-ec28-48e1-9c23-7988e0aa816a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:45.426173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:18:45.587550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c26ff-4fdd-4f81-b8b4-b387d7e703f7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:45.601932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c26ff-4fdd-4f81-b8b4-b387d7e703f7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:45.607275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1570e-5f66-4387-99da-84b8b8b85d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:45.607935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1570e-5f66-4387-99da-84b8b8b85d58 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:45.608221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1570e-5f66-4387-99da-84b8b8b85d58 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:45.608499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1570e-5f66-4387-99da-84b8b8b85d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:45.616918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1570e-5f66-4387-99da-84b8b8b85d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:45.618563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1570e-5f66-4387-99da-84b8b8b85d58 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:45.628430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 718/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:46.564674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eb7f6-bff4-425d-adc2-4ea01895cc39 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:46.565247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eb7f6-bff4-425d-adc2-4ea01895cc39 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:46.565247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eb7f6-bff4-425d-adc2-4ea01895cc39 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:46.567131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:46.572141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a2110-35b0-4864-a8f9-89655a04d28f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:46.572587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a2110-35b0-4864-a8f9-89655a04d28f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:46.572695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a2110-35b0-4864-a8f9-89655a04d28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:46.591608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b64a2110-35b0-4864-a8f9-89655a04d28f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:46.592783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b64a2110-35b0-4864-a8f9-89655a04d28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:46.600530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:18:46.749300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eb7f6-bff4-425d-adc2-4ea01895cc39 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:46.763731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eb7f6-bff4-425d-adc2-4ea01895cc39 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:46.769088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ae79c-2e01-40d3-9119-0aaf6ab77391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:46.769791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ae79c-2e01-40d3-9119-0aaf6ab77391 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:46.770073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ae79c-2e01-40d3-9119-0aaf6ab77391 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:46.770345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ae79c-2e01-40d3-9119-0aaf6ab77391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:46.778169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ae79c-2e01-40d3-9119-0aaf6ab77391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:46.779752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ae79c-2e01-40d3-9119-0aaf6ab77391 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:46.788869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 703/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:47.718901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24ab63-9bb4-4fa8-b5aa-87adc3007362 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:47.719473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24ab63-9bb4-4fa8-b5aa-87adc3007362 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:47.719582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24ab63-9bb4-4fa8-b5aa-87adc3007362 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:47.721122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:47.726620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdf95-5b0f-472f-a76f-f7a8257aead4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:47.727008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdf95-5b0f-472f-a76f-f7a8257aead4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:47.727181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdf95-5b0f-472f-a76f-f7a8257aead4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:47.745215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5cebdf95-5b0f-472f-a76f-f7a8257aead4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:47.746393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5cebdf95-5b0f-472f-a76f-f7a8257aead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:47.754886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:47.909985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24ab63-9bb4-4fa8-b5aa-87adc3007362 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:47.923978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24ab63-9bb4-4fa8-b5aa-87adc3007362 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:47.929475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a576636-1c5b-45ef-a5d1-5f89d662539d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:47.930098 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576636-1c5b-45ef-a5d1-5f89d662539d service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:47.930373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576636-1c5b-45ef-a5d1-5f89d662539d service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:47.930641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576636-1c5b-45ef-a5d1-5f89d662539d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:47.938460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a576636-1c5b-45ef-a5d1-5f89d662539d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:47.940064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a576636-1c5b-45ef-a5d1-5f89d662539d service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:47.949880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 721/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:48.933631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9505ad-0fc3-4dfb-8988-d1dcf7317bdc None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:48.934199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9505ad-0fc3-4dfb-8988-d1dcf7317bdc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:48.934199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9505ad-0fc3-4dfb-8988-d1dcf7317bdc None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:48.935705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:48.941210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc140-089e-47d3-a176-5bc42a0af3c2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:48.941652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc140-089e-47d3-a176-5bc42a0af3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:48.941748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc140-089e-47d3-a176-5bc42a0af3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:48.959544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d18dc140-089e-47d3-a176-5bc42a0af3c2 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:48.960745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d18dc140-089e-47d3-a176-5bc42a0af3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:48.968547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:49.117889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9505ad-0fc3-4dfb-8988-d1dcf7317bdc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:49.133003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9505ad-0fc3-4dfb-8988-d1dcf7317bdc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:49.138406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe463477-71f7-497c-b477-bcbe30a98dfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:49.139048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe463477-71f7-497c-b477-bcbe30a98dfd service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:49.139332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe463477-71f7-497c-b477-bcbe30a98dfd service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:49.139637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe463477-71f7-497c-b477-bcbe30a98dfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:49.147797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe463477-71f7-497c-b477-bcbe30a98dfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:49.149371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe463477-71f7-497c-b477-bcbe30a98dfd service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:49.158546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 706/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:50.009108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326dc87-a6c2-4784-8940-e1747e8dd34a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:50.009624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326dc87-a6c2-4784-8940-e1747e8dd34a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:50.009624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326dc87-a6c2-4784-8940-e1747e8dd34a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:50.010982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:50.015847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfe3de-1060-4a02-a963-339facf57e62 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:50.016311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfe3de-1060-4a02-a963-339facf57e62 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:50.016419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfe3de-1060-4a02-a963-339facf57e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:50.035110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-afbfe3de-1060-4a02-a963-339facf57e62 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:50.036346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-afbfe3de-1060-4a02-a963-339facf57e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:50.044147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:50.194853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326dc87-a6c2-4784-8940-e1747e8dd34a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:50.209614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326dc87-a6c2-4784-8940-e1747e8dd34a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:50.215972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e16740-3652-4602-8efe-fe6aa575e555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:50.217009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16740-3652-4602-8efe-fe6aa575e555 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:50.217521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16740-3652-4602-8efe-fe6aa575e555 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:50.218024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16740-3652-4602-8efe-fe6aa575e555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:50.227069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e16740-3652-4602-8efe-fe6aa575e555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:50.228681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e16740-3652-4602-8efe-fe6aa575e555 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:50.238218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 724/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:51.090287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b6bf-baa1-4903-8756-8caac5a519cd None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:51.090851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b6bf-baa1-4903-8756-8caac5a519cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:51.090851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b6bf-baa1-4903-8756-8caac5a519cd None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:51.092175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:51.097412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282505f-6145-4aca-b1b0-99450f588d40 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:51.097860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282505f-6145-4aca-b1b0-99450f588d40 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:51.097999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282505f-6145-4aca-b1b0-99450f588d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:51.116718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d282505f-6145-4aca-b1b0-99450f588d40 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:51.117915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d282505f-6145-4aca-b1b0-99450f588d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:51.125681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:51.274602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7b6bf-baa1-4903-8756-8caac5a519cd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:51.288647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7b6bf-baa1-4903-8756-8caac5a519cd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:51.293892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e52dbe-5080-4efd-928c-73aa53517cdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:51.294518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e52dbe-5080-4efd-928c-73aa53517cdf service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:51.294797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e52dbe-5080-4efd-928c-73aa53517cdf service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:51.295069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e52dbe-5080-4efd-928c-73aa53517cdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:51.303510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e52dbe-5080-4efd-928c-73aa53517cdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:51.305070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e52dbe-5080-4efd-928c-73aa53517cdf service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:51.314190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 709/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:52.161934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8267cd0-1ed5-434e-9669-7042c51a53ee None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:52.162502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8267cd0-1ed5-434e-9669-7042c51a53ee None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:52.162502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8267cd0-1ed5-434e-9669-7042c51a53ee None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:52.163897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:52.168840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec007-ec91-4475-95b6-09ed5f03555a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:52.169276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec007-ec91-4475-95b6-09ed5f03555a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:52.169395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec007-ec91-4475-95b6-09ed5f03555a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:52.187528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6dfec007-ec91-4475-95b6-09ed5f03555a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:52.188710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6dfec007-ec91-4475-95b6-09ed5f03555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:52.197243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:52.345934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8267cd0-1ed5-434e-9669-7042c51a53ee None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:52.360507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8267cd0-1ed5-434e-9669-7042c51a53ee None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:52.366784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f227cd0-c85e-4cfc-ab1f-f241062d051e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:52.367444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227cd0-c85e-4cfc-ab1f-f241062d051e service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:52.367720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227cd0-c85e-4cfc-ab1f-f241062d051e service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:52.367990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227cd0-c85e-4cfc-ab1f-f241062d051e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:52.376540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f227cd0-c85e-4cfc-ab1f-f241062d051e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:52.378244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f227cd0-c85e-4cfc-ab1f-f241062d051e service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:52.387552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 727/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:53.275370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec6f96-a0f9-4593-9156-9e42fed0a01f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:53.275942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec6f96-a0f9-4593-9156-9e42fed0a01f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:53.275942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec6f96-a0f9-4593-9156-9e42fed0a01f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:53.277296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:53.282155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5dbfe-397d-4445-8270-26e452ee1cbb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:53.282587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5dbfe-397d-4445-8270-26e452ee1cbb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:53.282731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5dbfe-397d-4445-8270-26e452ee1cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:53.301410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-efb5dbfe-397d-4445-8270-26e452ee1cbb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:53.302624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-efb5dbfe-397d-4445-8270-26e452ee1cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:53.310731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:53.462135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec6f96-a0f9-4593-9156-9e42fed0a01f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:53.476342 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec6f96-a0f9-4593-9156-9e42fed0a01f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:53.481617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da857087-b1d5-4cda-a62b-819321d4d3df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:53.482241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da857087-b1d5-4cda-a62b-819321d4d3df service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:53.482518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da857087-b1d5-4cda-a62b-819321d4d3df service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:53.482790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da857087-b1d5-4cda-a62b-819321d4d3df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:53.491891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da857087-b1d5-4cda-a62b-819321d4d3df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:53.493489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da857087-b1d5-4cda-a62b-819321d4d3df service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:53.503895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 712/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:55.689620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff255cf-04e7-4985-9c02-129ecad86dd8 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:55.690997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff255cf-04e7-4985-9c02-129ecad86dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:55.690997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff255cf-04e7-4985-9c02-129ecad86dd8 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:55.692482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:55.697831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf1d49-73c1-43c0-a82f-2e8ae892e66a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:55.698297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf1d49-73c1-43c0-a82f-2e8ae892e66a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:55.698391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf1d49-73c1-43c0-a82f-2e8ae892e66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:55.716960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bacf1d49-73c1-43c0-a82f-2e8ae892e66a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:55.718233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bacf1d49-73c1-43c0-a82f-2e8ae892e66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:55.726370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:55.768728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff255cf-04e7-4985-9c02-129ecad86dd8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:55.780791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff255cf-04e7-4985-9c02-129ecad86dd8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:55.784863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153ed6b-fae5-45c9-b59e-49a559d6d855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:55.785530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153ed6b-fae5-45c9-b59e-49a559d6d855 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:55.785807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153ed6b-fae5-45c9-b59e-49a559d6d855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:55.786079 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153ed6b-fae5-45c9-b59e-49a559d6d855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:55.795255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153ed6b-fae5-45c9-b59e-49a559d6d855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:55.796701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153ed6b-fae5-45c9-b59e-49a559d6d855 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:55.807250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 730/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:18:58.277717 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438c34c-f9fb-42c8-a636-74f57b902533 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:58.278252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438c34c-f9fb-42c8-a636-74f57b902533 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:58.278252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438c34c-f9fb-42c8-a636-74f57b902533 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:58.279597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:18:58.284754 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b87438-2ba7-4fb6-8eda-c07448298ca5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:58.285208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b87438-2ba7-4fb6-8eda-c07448298ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:58.285315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b87438-2ba7-4fb6-8eda-c07448298ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:58.302571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-44b87438-2ba7-4fb6-8eda-c07448298ca5 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:18:58.303914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-44b87438-2ba7-4fb6-8eda-c07448298ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:58.311571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:18:58.350205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438c34c-f9fb-42c8-a636-74f57b902533 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:18:58.351886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1438c34c-f9fb-42c8-a636-74f57b902533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:18:58.364612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438c34c-f9fb-42c8-a636-74f57b902533 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:18:58.368400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487cbf1-cb91-4c61-8166-90252ba81d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:18:58.369032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487cbf1-cb91-4c61-8166-90252ba81d2a service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:18:58.369315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487cbf1-cb91-4c61-8166-90252ba81d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:18:58.369587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487cbf1-cb91-4c61-8166-90252ba81d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:18:58.378808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487cbf1-cb91-4c61-8166-90252ba81d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:18:58.380258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487cbf1-cb91-4c61-8166-90252ba81d2a service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:18:58.389552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 715/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:00.907930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac7469-cac6-470c-8054-e06a4ec7ffe4 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:00.909085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac7469-cac6-470c-8054-e06a4ec7ffe4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:00.909085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac7469-cac6-470c-8054-e06a4ec7ffe4 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:00.910426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:00.915870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af76ad-611f-4ec6-a0f5-fb96e8273c65 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:00.916296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af76ad-611f-4ec6-a0f5-fb96e8273c65 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:00.916423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af76ad-611f-4ec6-a0f5-fb96e8273c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:00.933853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-47af76ad-611f-4ec6-a0f5-fb96e8273c65 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:00.935031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-47af76ad-611f-4ec6-a0f5-fb96e8273c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:00.943775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:00.984476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac7469-cac6-470c-8054-e06a4ec7ffe4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:00.995332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac7469-cac6-470c-8054-e06a4ec7ffe4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:00.998813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6c5db-0d71-4609-9abf-241f745958eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:00.999486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c5db-0d71-4609-9abf-241f745958eb service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:00.999768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c5db-0d71-4609-9abf-241f745958eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:01.000042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c5db-0d71-4609-9abf-241f745958eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:01.008839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6c5db-0d71-4609-9abf-241f745958eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:01.010272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6c5db-0d71-4609-9abf-241f745958eb service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:01.020036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 733/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:03.353117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9b44-4199-4a66-b31a-8f69b7d81ec9 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:03.353732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9b44-4199-4a66-b31a-8f69b7d81ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:03.354015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9b44-4199-4a66-b31a-8f69b7d81ec9 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:03.355441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:03.360801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d4772-d64d-4d4a-a1c3-92dc39f9bc70 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:03.361256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d4772-d64d-4d4a-a1c3-92dc39f9bc70 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:03.361354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d4772-d64d-4d4a-a1c3-92dc39f9bc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:03.380485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4e1d4772-d64d-4d4a-a1c3-92dc39f9bc70 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:03.381717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4e1d4772-d64d-4d4a-a1c3-92dc39f9bc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:03.389568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:03.427366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b9b44-4199-4a66-b31a-8f69b7d81ec9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:03.437944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b9b44-4199-4a66-b31a-8f69b7d81ec9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:03.442017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fb112-9707-4486-8b69-f6a1f5660778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:03.442646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fb112-9707-4486-8b69-f6a1f5660778 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:03.442929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fb112-9707-4486-8b69-f6a1f5660778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:03.443204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fb112-9707-4486-8b69-f6a1f5660778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:03.452080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fb112-9707-4486-8b69-f6a1f5660778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:03.453551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fb112-9707-4486-8b69-f6a1f5660778 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:03.462996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 718/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:05.794752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162725-7bb6-4986-8339-5b84b7700e79 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:05.795417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162725-7bb6-4986-8339-5b84b7700e79 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:05.795417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162725-7bb6-4986-8339-5b84b7700e79 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:05.796723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:05.802110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ba9f5-df83-4b1b-9a02-c68d2973521a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:05.802547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ba9f5-df83-4b1b-9a02-c68d2973521a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:05.802704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ba9f5-df83-4b1b-9a02-c68d2973521a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:05.820695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6c7ba9f5-df83-4b1b-9a02-c68d2973521a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:05.821889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6c7ba9f5-df83-4b1b-9a02-c68d2973521a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:05.829533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:19:05.867181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f162725-7bb6-4986-8339-5b84b7700e79 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:05.878289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f162725-7bb6-4986-8339-5b84b7700e79 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:05.881834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae2be1-d1fc-4c6f-9ad8-9163d082e348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:05.882474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae2be1-d1fc-4c6f-9ad8-9163d082e348 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:05.882756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae2be1-d1fc-4c6f-9ad8-9163d082e348 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:05.883032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae2be1-d1fc-4c6f-9ad8-9163d082e348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:05.891894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae2be1-d1fc-4c6f-9ad8-9163d082e348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:05.894042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae2be1-d1fc-4c6f-9ad8-9163d082e348 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:05.904192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 736/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:08.175846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886796-a607-4817-bc20-852f8e175916 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:08.176399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886796-a607-4817-bc20-852f8e175916 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:08.176399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886796-a607-4817-bc20-852f8e175916 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:08.177786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:08.183262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0269c-369e-4646-ad43-d1dc2b71ad6d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:08.183726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0269c-369e-4646-ad43-d1dc2b71ad6d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:08.183877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0269c-369e-4646-ad43-d1dc2b71ad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:08.201189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5ee0269c-369e-4646-ad43-d1dc2b71ad6d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:08.202369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5ee0269c-369e-4646-ad43-d1dc2b71ad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:08.210199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:08.248207 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30886796-a607-4817-bc20-852f8e175916 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:08.258953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30886796-a607-4817-bc20-852f8e175916 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:08.262550 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f98753-54eb-4ffd-ad23-0f963b1a0ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:08.263185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f98753-54eb-4ffd-ad23-0f963b1a0ecf service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:08.263506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f98753-54eb-4ffd-ad23-0f963b1a0ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:08.263787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f98753-54eb-4ffd-ad23-0f963b1a0ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:08.272703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f98753-54eb-4ffd-ad23-0f963b1a0ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:08.274123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f98753-54eb-4ffd-ad23-0f963b1a0ecf service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:08.283923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 721/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:10.562273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d29ab-a99a-4e05-8ae3-7b474222d41e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:10.562845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d29ab-a99a-4e05-8ae3-7b474222d41e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:10.562845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d29ab-a99a-4e05-8ae3-7b474222d41e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:10.564750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:10.570128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59919545-3377-4f1a-8942-7c71abbe5380 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:10.570600 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59919545-3377-4f1a-8942-7c71abbe5380 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:10.570709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59919545-3377-4f1a-8942-7c71abbe5380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:10.576653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=70 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:19:10.589867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-59919545-3377-4f1a-8942-7c71abbe5380 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:10.591060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-59919545-3377-4f1a-8942-7c71abbe5380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:10.598955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:19:10.638239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d29ab-a99a-4e05-8ae3-7b474222d41e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:10.643633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=73 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:19:10.649305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d29ab-a99a-4e05-8ae3-7b474222d41e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:10.652767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fc2db-953c-4cb8-858a-2b8fd6384ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:10.653401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fc2db-953c-4cb8-858a-2b8fd6384ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:10.653681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fc2db-953c-4cb8-858a-2b8fd6384ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:10.653955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fc2db-953c-4cb8-858a-2b8fd6384ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:10.663035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fc2db-953c-4cb8-858a-2b8fd6384ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:10.664516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fc2db-953c-4cb8-858a-2b8fd6384ef2 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:10.674278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 739/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:12.924861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0531a9-741e-4406-a928-1a016494408f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:12.925400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0531a9-741e-4406-a928-1a016494408f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:12.925400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0531a9-741e-4406-a928-1a016494408f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:12.926758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:12.932015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e805f-da61-4251-a554-718bc2de3faa None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:12.932468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e805f-da61-4251-a554-718bc2de3faa None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:12.932579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e805f-da61-4251-a554-718bc2de3faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:12.950945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c41e805f-da61-4251-a554-718bc2de3faa None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:12.952160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c41e805f-da61-4251-a554-718bc2de3faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:12.960288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:12.997451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0531a9-741e-4406-a928-1a016494408f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:13.009593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0531a9-741e-4406-a928-1a016494408f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:13.013046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7666b84-0a4b-402c-b38f-b74f740adcec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:13.013707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7666b84-0a4b-402c-b38f-b74f740adcec service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:13.013988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7666b84-0a4b-402c-b38f-b74f740adcec service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:13.014258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7666b84-0a4b-402c-b38f-b74f740adcec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:13.023526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7666b84-0a4b-402c-b38f-b74f740adcec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:13.024979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7666b84-0a4b-402c-b38f-b74f740adcec service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:13.034415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 724/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:15.266407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8613a-1cce-45b0-bb90-461c1944c5fe None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:15.267002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8613a-1cce-45b0-bb90-461c1944c5fe None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:15.267002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8613a-1cce-45b0-bb90-461c1944c5fe None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:15.268495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:15.273828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121dc1-6278-498e-bc05-6de2a1f66bf1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:15.274252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121dc1-6278-498e-bc05-6de2a1f66bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:15.274370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121dc1-6278-498e-bc05-6de2a1f66bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:15.292376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f0121dc1-6278-498e-bc05-6de2a1f66bf1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:15.293550 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f0121dc1-6278-498e-bc05-6de2a1f66bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:15.301859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:15.338907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8613a-1cce-45b0-bb90-461c1944c5fe None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:15.350080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8613a-1cce-45b0-bb90-461c1944c5fe None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:15.353554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cb815-9231-4e86-9ec3-e07e488c91ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:15.354184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb815-9231-4e86-9ec3-e07e488c91ca service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:15.354465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb815-9231-4e86-9ec3-e07e488c91ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:15.354739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb815-9231-4e86-9ec3-e07e488c91ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:15.364235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cb815-9231-4e86-9ec3-e07e488c91ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:15.365934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cb815-9231-4e86-9ec3-e07e488c91ca service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:15.375135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 742/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:17.602263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01230-e5c6-4b62-aaa8-a2be43ecdd8c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:17.602835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01230-e5c6-4b62-aaa8-a2be43ecdd8c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:17.602835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01230-e5c6-4b62-aaa8-a2be43ecdd8c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:17.604243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:17.609517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4d5-8094-4aa4-837f-4f0b738e82f0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:17.609904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4d5-8094-4aa4-837f-4f0b738e82f0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:17.610064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4d5-8094-4aa4-837f-4f0b738e82f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:17.628900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f636e4d5-8094-4aa4-837f-4f0b738e82f0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:17.630094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f636e4d5-8094-4aa4-837f-4f0b738e82f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:17.637713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:17.674775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01230-e5c6-4b62-aaa8-a2be43ecdd8c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:17.685272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01230-e5c6-4b62-aaa8-a2be43ecdd8c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:17.688747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962165e-7f5b-4f06-b2a3-1da08570ec28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:17.689968 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962165e-7f5b-4f06-b2a3-1da08570ec28 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:17.690247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962165e-7f5b-4f06-b2a3-1da08570ec28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:17.690521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962165e-7f5b-4f06-b2a3-1da08570ec28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:17.699309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962165e-7f5b-4f06-b2a3-1da08570ec28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:17.700751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962165e-7f5b-4f06-b2a3-1da08570ec28 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:17.710542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 727/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:27.623049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=17 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:19:27.706467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=17 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:19:50.830965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df8f23-2ae2-4d01-88a9-051f088e2042 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:50.831579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df8f23-2ae2-4d01-88a9-051f088e2042 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:50.831579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df8f23-2ae2-4d01-88a9-051f088e2042 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:50.832965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 744/1471] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:50.838339 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d83ac2-aa27-4cd6-8e6f-4b09e8485a1d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:50.838748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d83ac2-aa27-4cd6-8e6f-4b09e8485a1d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:50.838884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d83ac2-aa27-4cd6-8e6f-4b09e8485a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:50.856698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c0d83ac2-aa27-4cd6-8e6f-4b09e8485a1d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:50.857872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c0d83ac2-aa27-4cd6-8e6f-4b09e8485a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:50.866400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 728/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:50.914238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40cc1-272f-41cd-b0ae-f3a8aee8e4b6 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:50.914528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40cc1-272f-41cd-b0ae-f3a8aee8e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:50.914805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40cc1-272f-41cd-b0ae-f3a8aee8e4b6 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:50.916213 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 745/1473] 10.222.0.33 () {54 vars in 962 bytes} [Sat May 28 17:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:50.921007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdad2bf-38d6-4d24-ade9-3b2c65431008 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:50.921294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdad2bf-38d6-4d24-ade9-3b2c65431008 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:50.921572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdad2bf-38d6-4d24-ade9-3b2c65431008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:50.940162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5fdad2bf-38d6-4d24-ade9-3b2c65431008 None None] MFA Rules not processed for user `58b8dd344e8e43ec9cec222737f98ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:50.941343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5fdad2bf-38d6-4d24-ade9-3b2c65431008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:50.947695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 729/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Sat May 28 17:19:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:19:50.955553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40cc1-272f-41cd-b0ae-f3a8aee8e4b6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:50.966029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40cc1-272f-41cd-b0ae-f3a8aee8e4b6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:50.969657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed03457-4284-4e98-b139-5ea949e1f8fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:50.970287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03457-4284-4e98-b139-5ea949e1f8fb service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:50.970569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03457-4284-4e98-b139-5ea949e1f8fb service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:50.970843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03457-4284-4e98-b139-5ea949e1f8fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:50.979737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed03457-4284-4e98-b139-5ea949e1f8fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:50.981120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed03457-4284-4e98-b139-5ea949e1f8fb service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:50.990773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 746/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:53.301402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8a915-a77b-4b9e-bfd1-8972ba145f4d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:53.301972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8a915-a77b-4b9e-bfd1-8972ba145f4d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:53.301972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8a915-a77b-4b9e-bfd1-8972ba145f4d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:53.303308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 730/1476] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:53.308464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcdfad-9d10-498f-9147-a1222b239c8e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:53.308894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcdfad-9d10-498f-9147-a1222b239c8e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:53.309054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcdfad-9d10-498f-9147-a1222b239c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:53.326823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6adcdfad-9d10-498f-9147-a1222b239c8e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:53.328557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6adcdfad-9d10-498f-9147-a1222b239c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:53.336430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 747/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:53.359556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8a915-a77b-4b9e-bfd1-8972ba145f4d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:53.371981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8a915-a77b-4b9e-bfd1-8972ba145f4d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:53.376569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905b0c7-97bc-4a82-9470-c8d02147e5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:53.377203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b0c7-97bc-4a82-9470-c8d02147e5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:53.377480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b0c7-97bc-4a82-9470-c8d02147e5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:53.377755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b0c7-97bc-4a82-9470-c8d02147e5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:53.378478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905b0c7-97bc-4a82-9470-c8d02147e5f2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:53.380249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-8905b0c7-97bc-4a82-9470-c8d02147e5f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:19:53.380249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 28 17:19:53.380249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: May 28 17:19:53.381409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905b0c7-97bc-4a82-9470-c8d02147e5f2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:53.401166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 731/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 17:19:53 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:19:55.636844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890abb4-0c3e-4320-a928-d8874980d9b7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:55.637411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890abb4-0c3e-4320-a928-d8874980d9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:55.637411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890abb4-0c3e-4320-a928-d8874980d9b7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:55.638753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 748/1479] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:55.644761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4109f-f842-4acf-b0eb-d76dc5337aa5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:55.645175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4109f-f842-4acf-b0eb-d76dc5337aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:55.645344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4109f-f842-4acf-b0eb-d76dc5337aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:55.663418 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0ce4109f-f842-4acf-b0eb-d76dc5337aa5 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:55.664611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0ce4109f-f842-4acf-b0eb-d76dc5337aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:55.672683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 732/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:55.717190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890abb4-0c3e-4320-a928-d8874980d9b7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:55.732217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890abb4-0c3e-4320-a928-d8874980d9b7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:55.737566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13cdd2-c08b-4955-968a-4c778905a2f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:55.738204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13cdd2-c08b-4955-968a-4c778905a2f5 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:55.738487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13cdd2-c08b-4955-968a-4c778905a2f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:55.738762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13cdd2-c08b-4955-968a-4c778905a2f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:55.747131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13cdd2-c08b-4955-968a-4c778905a2f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:55.748753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13cdd2-c08b-4955-968a-4c778905a2f5 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:55.758643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 749/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:19:58.130513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d808-53f9-4f6f-a62f-bdb75478835c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:58.131030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d808-53f9-4f6f-a62f-bdb75478835c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:58.131147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d808-53f9-4f6f-a62f-bdb75478835c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:58.132474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 733/1482] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:19:58.137723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb2a8e-0716-4fbf-85f5-cdc5f8164ef8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:58.138162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb2a8e-0716-4fbf-85f5-cdc5f8164ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:58.138299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb2a8e-0716-4fbf-85f5-cdc5f8164ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:58.157216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9ccb2a8e-0716-4fbf-85f5-cdc5f8164ef8 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:19:58.158421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9ccb2a8e-0716-4fbf-85f5-cdc5f8164ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:19:58.166206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 750/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:19:58.201726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d808-53f9-4f6f-a62f-bdb75478835c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:19:58.213170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d808-53f9-4f6f-a62f-bdb75478835c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:19:58.216588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f6794-f7eb-45bd-81b5-d326aa22c3ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:19:58.217212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6794-f7eb-45bd-81b5-d326aa22c3ac service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:19:58.217493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6794-f7eb-45bd-81b5-d326aa22c3ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:19:58.217762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6794-f7eb-45bd-81b5-d326aa22c3ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:19:58.227152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f6794-f7eb-45bd-81b5-d326aa22c3ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:19:58.228602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f6794-f7eb-45bd-81b5-d326aa22c3ac service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:19:58.237943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 734/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:20:00.510501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57a8d-0aa1-4f00-9a6c-127ec8580f23 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:00.511043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57a8d-0aa1-4f00-9a6c-127ec8580f23 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:00.511043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57a8d-0aa1-4f00-9a6c-127ec8580f23 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:00.512501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 751/1485] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:20:00.518477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7d16-8eea-4b6c-aaf7-dba160e6bf50 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:00.518889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7d16-8eea-4b6c-aaf7-dba160e6bf50 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:00.519033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7d16-8eea-4b6c-aaf7-dba160e6bf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:00.536616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-da4f7d16-8eea-4b6c-aaf7-dba160e6bf50 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:20:00.537806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-da4f7d16-8eea-4b6c-aaf7-dba160e6bf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:20:00.545393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 735/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:20:00.583400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c57a8d-0aa1-4f00-9a6c-127ec8580f23 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:20:00.594127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c57a8d-0aa1-4f00-9a6c-127ec8580f23 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:20:00.597772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee84d4f-4e53-42a8-9674-4613b8e665a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:20:00.598403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84d4f-4e53-42a8-9674-4613b8e665a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:00.598701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84d4f-4e53-42a8-9674-4613b8e665a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:00.598978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84d4f-4e53-42a8-9674-4613b8e665a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:00.607879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee84d4f-4e53-42a8-9674-4613b8e665a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:20:00.609433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee84d4f-4e53-42a8-9674-4613b8e665a2 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:20:00.619207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 752/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:20:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:20:02.951279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791417f-869a-4584-9df2-4b1723d6987e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:02.951914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791417f-869a-4584-9df2-4b1723d6987e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:02.951914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791417f-869a-4584-9df2-4b1723d6987e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:02.953954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 736/1488] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:20:02.959165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9a03-a35a-48e1-926a-70a4807448eb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:02.959601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9a03-a35a-48e1-926a-70a4807448eb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:02.959753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9a03-a35a-48e1-926a-70a4807448eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:02.977717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-205f9a03-a35a-48e1-926a-70a4807448eb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:20:02.978842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-205f9a03-a35a-48e1-926a-70a4807448eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:20:02.986835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 753/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:20:03.029563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791417f-869a-4584-9df2-4b1723d6987e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:20:03.040236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791417f-869a-4584-9df2-4b1723d6987e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:20:03.043781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa39212-d246-4ded-a211-c2edab259bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:20:03.044411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa39212-d246-4ded-a211-c2edab259bac service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:03.044690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa39212-d246-4ded-a211-c2edab259bac service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:03.044963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa39212-d246-4ded-a211-c2edab259bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:03.053788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa39212-d246-4ded-a211-c2edab259bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:20:03.055202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa39212-d246-4ded-a211-c2edab259bac service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:20:03.064955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 737/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:20:05.465548 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810326-1bd5-469a-8002-dedea38d4f9c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:05.466115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810326-1bd5-469a-8002-dedea38d4f9c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:05.466115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810326-1bd5-469a-8002-dedea38d4f9c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:05.467493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 754/1491] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 17:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:20:05.473552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a48db-c737-4038-8130-e43f27c03b3a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:05.474013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a48db-c737-4038-8130-e43f27c03b3a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:05.474133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a48db-c737-4038-8130-e43f27c03b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:05.492580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4e3a48db-c737-4038-8130-e43f27c03b3a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:20:05.493972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4e3a48db-c737-4038-8130-e43f27c03b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:20:05.501826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 738/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 17:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:20:05.540878 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb810326-1bd5-469a-8002-dedea38d4f9c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:20:05.551763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb810326-1bd5-469a-8002-dedea38d4f9c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:20:05.555146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eefca-cdb0-41df-beb6-f09cd78c1812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:20:05.555808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eefca-cdb0-41df-beb6-f09cd78c1812 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:20:05.556088 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eefca-cdb0-41df-beb6-f09cd78c1812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:20:05.556361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eefca-cdb0-41df-beb6-f09cd78c1812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:20:05.565196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eefca-cdb0-41df-beb6-f09cd78c1812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:20:05.566616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eefca-cdb0-41df-beb6-f09cd78c1812 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:20:05.575681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 755/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:20:15.486878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=33 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:20:15.571681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=33 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:21:45.586649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f8e56-641c-453e-84f1-444164872731 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.587247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f8e56-641c-453e-84f1-444164872731 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.587247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f8e56-641c-453e-84f1-444164872731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:45.605555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-425f8e56-641c-453e-84f1-444164872731 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:21:45.607398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-425f8e56-641c-453e-84f1-444164872731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:21:45.615156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 739/1494] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:21:45.624568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb4c8c-6beb-4562-b22a-e0ce2e5b6772 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.625083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb4c8c-6beb-4562-b22a-e0ce2e5b6772 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.625186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb4c8c-6beb-4562-b22a-e0ce2e5b6772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:45.643552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8edb4c8c-6beb-4562-b22a-e0ce2e5b6772 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:21:45.644738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8edb4c8c-6beb-4562-b22a-e0ce2e5b6772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:21:45.652401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 756/1495] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:21:45.657562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f8e56-641c-453e-84f1-444164872731 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:45.672070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f8e56-641c-453e-84f1-444164872731 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:45.676623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:45.677308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.677597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.677876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:45.678603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:21:45.678909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:45.679978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 17:21:45.679978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 28 17:21:45.679978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: May 28 17:21:45.680850 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:45.697465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 740/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:21:45.705470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb4c8c-6beb-4562-b22a-e0ce2e5b6772 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:45.714468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb4c8c-6beb-4562-b22a-e0ce2e5b6772 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:45.719679 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:45.720331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.720604 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.720872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:45.721973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:45.723442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:21:45.723442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: warnings.warn(msg) May 28 17:21:45.723442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: May 28 17:21:45.724634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:45.761539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 757/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:21:45 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:21:45.765703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:45.775478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30292761-ca8a-4bfc-a95f-3cf17c9f9bf2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:45.831861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:45.832538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.832846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.833117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:45.833799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:45.834803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:21:45.834803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 28 17:21:45.834803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: May 28 17:21:45.836032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:45.857516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 741/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:21:45 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:21:45.861506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:45.872110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8bad1-be4d-4128-a7b4-23ffb1bbcf52 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:45.876547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:45.877196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.877475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.877745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:45.878458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:45.879510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 17:21:45.879510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: warnings.warn(msg) May 28 17:21:45.879510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: May 28 17:21:45.880364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:45.893583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 758/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:21:45.897627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:45.908065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720c10a-5b67-4ba6-a3e4-7970cce6fcb5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:45.912505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde9344-23aa-44fd-b3bc-56e8b6fda7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:45.913153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde9344-23aa-44fd-b3bc-56e8b6fda7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.913433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde9344-23aa-44fd-b3bc-56e8b6fda7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.913710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde9344-23aa-44fd-b3bc-56e8b6fda7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:45.914431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde9344-23aa-44fd-b3bc-56e8b6fda7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:45.915462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-1dde9344-23aa-44fd-b3bc-56e8b6fda7ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 17:21:45.915462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 28 17:21:45.915462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: May 28 17:21:45.916332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde9344-23aa-44fd-b3bc-56e8b6fda7ad admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:45.929383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 742/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:21:45.933435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:45.943273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594045a-4d26-4359-a87f-cc4999738236 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:45.947943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1ab13-2096-4230-8d09-b644cc25d15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:45.948868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1ab13-2096-4230-8d09-b644cc25d15b admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.949143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1ab13-2096-4230-8d09-b644cc25d15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.949417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1ab13-2096-4230-8d09-b644cc25d15b admin admin] PATH_INFO: `/v3/projects/70d63ad291644b309180cd046a9e0c24/users/a525f707919344209d9f2908c18a823c/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:45.965061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1ab13-2096-4230-8d09-b644cc25d15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=70d63ad291644b309180cd046a9e0c24, user_id=a525f707919344209d9f2908c18a823c, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:45.966493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-33f1ab13-2096-4230-8d09-b644cc25d15b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:21:45.966493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: warnings.warn(msg) May 28 17:21:45.966493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: May 28 17:21:45.968443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1ab13-2096-4230-8d09-b644cc25d15b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:45.982041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 759/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:21:45 2022] PUT /identity/v3/projects/70d63ad291644b309180cd046a9e0c24/users/a525f707919344209d9f2908c18a823c/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:21:45.993682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ab511-89fb-4030-9cc5-12952a3ab25e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:45.993979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ab511-89fb-4030-9cc5-12952a3ab25e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:45.994265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ab511-89fb-4030-9cc5-12952a3ab25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.020598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6a1ab511-89fb-4030-9cc5-12952a3ab25e None None] MFA Rules not processed for user `a525f707919344209d9f2908c18a823c`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:21:46.021814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6a1ab511-89fb-4030-9cc5-12952a3ab25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:21:46.084887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 743/1502] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:21:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:21:46.097940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88099d5d-0669-4130-a559-460326392383 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:46.098266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88099d5d-0669-4130-a559-460326392383 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:46.098543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88099d5d-0669-4130-a559-460326392383 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.099953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 760/1503] 10.222.0.33 () {54 vars in 963 bytes} [Sat May 28 17:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:21:46.104646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7cd5-fc5d-4dc6-9ce3-d5dcebe30890 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:46.104971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7cd5-fc5d-4dc6-9ce3-d5dcebe30890 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:46.105252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7cd5-fc5d-4dc6-9ce3-d5dcebe30890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.124265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2eaa7cd5-fc5d-4dc6-9ce3-d5dcebe30890 None None] MFA Rules not processed for user `57af250e151241f4afc9b38d51a5fbdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:21:46.125401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2eaa7cd5-fc5d-4dc6-9ce3-d5dcebe30890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:21:46.167633 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:21:46.176497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 744/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Sat May 28 17:21:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:21:46.184957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88099d5d-0669-4130-a559-460326392383 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:46.195296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88099d5d-0669-4130-a559-460326392383 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:46.198757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88677c9-266b-4fdb-83a0-aa1ffc0d9bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:46.199407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88677c9-266b-4fdb-83a0-aa1ffc0d9bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:46.199728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88677c9-266b-4fdb-83a0-aa1ffc0d9bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:46.200068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88677c9-266b-4fdb-83a0-aa1ffc0d9bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.208243 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88677c9-266b-4fdb-83a0-aa1ffc0d9bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:46.209667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88677c9-266b-4fdb-83a0-aa1ffc0d9bd8 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:46.219046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 761/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:21:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:21:46.261271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d262dc-e190-4d0c-9b11-51b66dd41c85 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:46.261557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d262dc-e190-4d0c-9b11-51b66dd41c85 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:46.261831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d262dc-e190-4d0c-9b11-51b66dd41c85 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.263113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 745/1506] 10.222.0.33 () {52 vars in 829 bytes} [Sat May 28 17:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:21:46.266660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88677c9-266b-4fdb-83a0-aa1ffc0d9bd8 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:46.327327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88677c9-266b-4fdb-83a0-aa1ffc0d9bd8 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:46.331858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:46.332723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:46.333004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:46.333281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] PATH_INFO: `/v3/users/a525f707919344209d9f2908c18a823c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.334463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a525f707919344209d9f2908c18a823c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:46.335520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:21:46.335520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: warnings.warn(msg) May 28 17:21:46.335520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: May 28 17:21:46.336713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:46.399319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a525f707919344209d9f2908c18a823c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:46.399927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a525f707919344209d9f2908c18a823c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:46.413027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a525f707919344209d9f2908c18a823c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:46.431725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a525f707919344209d9f2908c18a823c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:46.432524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a525f707919344209d9f2908c18a823c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:46.436595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:21:46.448519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 762/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:21:46 2022] DELETE /identity/v3/users/a525f707919344209d9f2908c18a823c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:21:46.452965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4dea8-e403-4ecd-a18b-d14c1f701d56 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:46.453276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4dea8-e403-4ecd-a18b-d14c1f701d56 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:46.453553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4dea8-e403-4ecd-a18b-d14c1f701d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.472388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c0e4dea8-e403-4ecd-a18b-d14c1f701d56 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:21:46.473610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c0e4dea8-e403-4ecd-a18b-d14c1f701d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:21:46.535039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 746/1508] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:21:46.545087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:46.546497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:21:46.603516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506c0a-01d4-4be3-8e83-c828bb854568 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:46.607120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaaaa9-ba88-41f7-8424-36b7a883bb57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:46.607791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaaaa9-ba88-41f7-8424-36b7a883bb57 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:46.608080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaaaa9-ba88-41f7-8424-36b7a883bb57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:46.608359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaaaa9-ba88-41f7-8424-36b7a883bb57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.617409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaaaa9-ba88-41f7-8424-36b7a883bb57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:46.618851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaaaa9-ba88-41f7-8424-36b7a883bb57 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:46.628294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 763/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:21:46.948904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4dea8-e403-4ecd-a18b-d14c1f701d56 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:21:46.950471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c0e4dea8-e403-4ecd-a18b-d14c1f701d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:21:46.962225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4dea8-e403-4ecd-a18b-d14c1f701d56 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:21:46.968177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:21:46.969665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:21:46.970462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:21:46.970750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] PATH_INFO: `/v3/projects/70d63ad291644b309180cd046a9e0c24` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:21:46.972004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70d63ad291644b309180cd046a9e0c24)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:21:46.973019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:21:46.973019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 28 17:21:46.973019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: May 28 17:21:46.975539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:21:47.037200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70d63ad291644b309180cd046a9e0c24', 'request_id': 'req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:47.048475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70d63ad291644b309180cd046a9e0c24', 'request_id': 'req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:47.062082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70d63ad291644b309180cd046a9e0c24', 'request_id': 'req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:47.073271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] The token cache is being invalidate because project 70d63ad291644b309180cd046a9e0c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:21:47.073558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:47.074295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c25de308-9f8f-4efc-9ab3-50d7d6c2dc9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:21:47.076025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 747/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:21:46 2022] DELETE /identity/v3/projects/70d63ad291644b309180cd046a9e0c24 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:21:56.624182 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:21:57.070275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:23:25.703191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd870462-739d-40a5-9c28-9b0641a1ba5d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:25.703809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd870462-739d-40a5-9c28-9b0641a1ba5d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:25.703809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd870462-739d-40a5-9c28-9b0641a1ba5d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:25.705147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:23:25.709943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37324cf0-6656-4ff6-bf65-de64958ca3a1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:25.710351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37324cf0-6656-4ff6-bf65-de64958ca3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:25.710503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37324cf0-6656-4ff6-bf65-de64958ca3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:25.731795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-37324cf0-6656-4ff6-bf65-de64958ca3a1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:25.733016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-37324cf0-6656-4ff6-bf65-de64958ca3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:25.790992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:23:25.957441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd870462-739d-40a5-9c28-9b0641a1ba5d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:25.959151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bd870462-739d-40a5-9c28-9b0641a1ba5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:26.033778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd870462-739d-40a5-9c28-9b0641a1ba5d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:26.039170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835ccfa-a568-48ff-9c23-8692d5c592f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:26.039835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ccfa-a568-48ff-9c23-8692d5c592f4 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:26.040123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ccfa-a568-48ff-9c23-8692d5c592f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:26.040397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ccfa-a568-48ff-9c23-8692d5c592f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:26.049497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835ccfa-a568-48ff-9c23-8692d5c592f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:26.051091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835ccfa-a568-48ff-9c23-8692d5c592f4 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:26.060826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 765/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:23:26.953187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75064829-5c00-4f9c-a35e-94c82eeab741 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:26.953767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75064829-5c00-4f9c-a35e-94c82eeab741 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:26.953767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75064829-5c00-4f9c-a35e-94c82eeab741 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:26.955166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:23:26.960042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccf26a-3829-4b4f-b458-899e61791c4d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:26.960504 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccf26a-3829-4b4f-b458-899e61791c4d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:26.960622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccf26a-3829-4b4f-b458-899e61791c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:26.979095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e0ccf26a-3829-4b4f-b458-899e61791c4d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:26.980253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e0ccf26a-3829-4b4f-b458-899e61791c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:26.987959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:27.142625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75064829-5c00-4f9c-a35e-94c82eeab741 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:27.157715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75064829-5c00-4f9c-a35e-94c82eeab741 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:27.162955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b3cd1-749b-4927-a8f6-ca4c3eb1eba4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:27.163612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b3cd1-749b-4927-a8f6-ca4c3eb1eba4 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:27.163894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b3cd1-749b-4927-a8f6-ca4c3eb1eba4 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:27.164170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b3cd1-749b-4927-a8f6-ca4c3eb1eba4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:27.172145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b3cd1-749b-4927-a8f6-ca4c3eb1eba4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:27.173722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b3cd1-749b-4927-a8f6-ca4c3eb1eba4 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:27.182878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 750/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:23:27.450272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900812-4443-4bda-b5c3-f9725ebe7a03 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:27.450565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900812-4443-4bda-b5c3-f9725ebe7a03 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:27.450849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900812-4443-4bda-b5c3-f9725ebe7a03 None None] PATH_INFO: `/v3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:27.452220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 767/1517] 10.222.0.16 () {54 vars in 925 bytes} [Sat May 28 17:23:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:23:27.458336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b168364-886f-45b9-af2a-3547e3ba7755 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:27.458621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b168364-886f-45b9-af2a-3547e3ba7755 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:27.458894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b168364-886f-45b9-af2a-3547e3ba7755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:27.476907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4b168364-886f-45b9-af2a-3547e3ba7755 None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:27.478148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4b168364-886f-45b9-af2a-3547e3ba7755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:27.540907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 751/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Sat May 28 17:23:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:36.557114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094b60e-b0ce-457c-969f-366795ad2555 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:36.557848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094b60e-b0ce-457c-969f-366795ad2555 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:36.557848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094b60e-b0ce-457c-969f-366795ad2555 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:36.559232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 768/1519] 10.222.0.33 () {54 vars in 960 bytes} [Sat May 28 17:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:23:36.564343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4023ce-5f3c-4570-b571-2f9f4466251e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:36.564767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4023ce-5f3c-4570-b571-2f9f4466251e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:36.564885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4023ce-5f3c-4570-b571-2f9f4466251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:36.583297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fe4023ce-5f3c-4570-b571-2f9f4466251e None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:36.584541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fe4023ce-5f3c-4570-b571-2f9f4466251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:36.594154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 752/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Sat May 28 17:23:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:36.601600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094b60e-b0ce-457c-969f-366795ad2555 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:36.615450 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094b60e-b0ce-457c-969f-366795ad2555 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:36.621217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b9c7-d92c-4076-a908-fc21f7206e5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:36.621843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b9c7-d92c-4076-a908-fc21f7206e5d service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:36.622123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b9c7-d92c-4076-a908-fc21f7206e5d service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:36.622403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b9c7-d92c-4076-a908-fc21f7206e5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:36.630765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b9c7-d92c-4076-a908-fc21f7206e5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:36.632409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b9c7-d92c-4076-a908-fc21f7206e5d service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:36.642416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 769/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:23:38.497025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0fa5-02a0-4900-b4f2-5ccd98401467 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:38.497662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0fa5-02a0-4900-b4f2-5ccd98401467 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:38.497662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0fa5-02a0-4900-b4f2-5ccd98401467 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:38.498959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:23:38.503725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636449-9ec7-4af6-a523-1bb9e895d2a0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:38.504184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636449-9ec7-4af6-a523-1bb9e895d2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:38.504290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636449-9ec7-4af6-a523-1bb9e895d2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:38.525646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ec636449-9ec7-4af6-a523-1bb9e895d2a0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:38.527440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ec636449-9ec7-4af6-a523-1bb9e895d2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:38.535198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:38.696997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b0fa5-02a0-4900-b4f2-5ccd98401467 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:38.711324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b0fa5-02a0-4900-b4f2-5ccd98401467 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:38.717415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43dc2d-8ca8-4a8a-b9e3-b67341814c13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:38.718062 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43dc2d-8ca8-4a8a-b9e3-b67341814c13 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:38.718343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43dc2d-8ca8-4a8a-b9e3-b67341814c13 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:38.718618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43dc2d-8ca8-4a8a-b9e3-b67341814c13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:38.727139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43dc2d-8ca8-4a8a-b9e3-b67341814c13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:38.728768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43dc2d-8ca8-4a8a-b9e3-b67341814c13 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:38.738997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 754/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:23:39.624922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591d15-c267-4be4-a6ec-cc6bd6782636 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:39.625526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591d15-c267-4be4-a6ec-cc6bd6782636 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:39.625526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591d15-c267-4be4-a6ec-cc6bd6782636 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:39.626848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 17:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:23:39.631746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d703ef-6b51-4014-9dbc-f7067df8bd7f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:39.632178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d703ef-6b51-4014-9dbc-f7067df8bd7f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:39.632331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d703ef-6b51-4014-9dbc-f7067df8bd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:39.650414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-82d703ef-6b51-4014-9dbc-f7067df8bd7f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:39.652217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-82d703ef-6b51-4014-9dbc-f7067df8bd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:39.659923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 17:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:39.809959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55591d15-c267-4be4-a6ec-cc6bd6782636 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:39.824709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55591d15-c267-4be4-a6ec-cc6bd6782636 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:39.830156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94493f32-c66e-428a-a4f6-0b14e6e20f86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:39.830780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493f32-c66e-428a-a4f6-0b14e6e20f86 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:39.831058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493f32-c66e-428a-a4f6-0b14e6e20f86 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:39.831326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493f32-c66e-428a-a4f6-0b14e6e20f86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:39.839987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94493f32-c66e-428a-a4f6-0b14e6e20f86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:39.841620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94493f32-c66e-428a-a4f6-0b14e6e20f86 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:39.850967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 772/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:23:43.777822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a9f44-7064-4c05-9bd0-cd7dc39fa84b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:43.778367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a9f44-7064-4c05-9bd0-cd7dc39fa84b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:43.778367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a9f44-7064-4c05-9bd0-cd7dc39fa84b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:43.779792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:23:43.784892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ddd58-ce02-4ec0-999a-8f6b66c6e3b8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:43.785314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ddd58-ce02-4ec0-999a-8f6b66c6e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:43.785449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ddd58-ce02-4ec0-999a-8f6b66c6e3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:43.803074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5f2ddd58-ce02-4ec0-999a-8f6b66c6e3b8 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:43.804310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5f2ddd58-ce02-4ec0-999a-8f6b66c6e3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:43.813560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:43.850691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a9f44-7064-4c05-9bd0-cd7dc39fa84b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:43.852495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6d1a9f44-7064-4c05-9bd0-cd7dc39fa84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:43.910293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a9f44-7064-4c05-9bd0-cd7dc39fa84b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:43.913789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70486160-390f-4924-a451-2dcf3d499660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:43.914425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70486160-390f-4924-a451-2dcf3d499660 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:43.914707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70486160-390f-4924-a451-2dcf3d499660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:43.914982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70486160-390f-4924-a451-2dcf3d499660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:43.923725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70486160-390f-4924-a451-2dcf3d499660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:43.925128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70486160-390f-4924-a451-2dcf3d499660 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:43.929922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:23:43.935464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 757/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:23:46.154341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b93f8-611c-4bb3-b166-5cb2609e71e1 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:46.154867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b93f8-611c-4bb3-b166-5cb2609e71e1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:46.154867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b93f8-611c-4bb3-b166-5cb2609e71e1 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:46.156219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:23:46.161706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce65d3-f3b8-44ff-8104-6dbfa26d18ee None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:46.162203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce65d3-f3b8-44ff-8104-6dbfa26d18ee None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:46.162305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce65d3-f3b8-44ff-8104-6dbfa26d18ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:46.182564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4cce65d3-f3b8-44ff-8104-6dbfa26d18ee None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:46.183820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4cce65d3-f3b8-44ff-8104-6dbfa26d18ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:46.191868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:46.231193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b93f8-611c-4bb3-b166-5cb2609e71e1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:46.242493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b93f8-611c-4bb3-b166-5cb2609e71e1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:46.246083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee00a9-17aa-4e57-81b3-b85c82e1b15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:46.246724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee00a9-17aa-4e57-81b3-b85c82e1b15e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:46.247023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee00a9-17aa-4e57-81b3-b85c82e1b15e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:46.247298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee00a9-17aa-4e57-81b3-b85c82e1b15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:46.256280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee00a9-17aa-4e57-81b3-b85c82e1b15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:46.257730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee00a9-17aa-4e57-81b3-b85c82e1b15e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:46.267136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 775/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:23:49.953483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e4056-3b4c-4d3f-91a0-966ceeb46d75 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:49.954047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e4056-3b4c-4d3f-91a0-966ceeb46d75 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:49.954047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e4056-3b4c-4d3f-91a0-966ceeb46d75 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:49.955414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:23:49.960625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55040-766c-4960-856d-3c795f85bbc0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:49.961036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55040-766c-4960-856d-3c795f85bbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:49.961178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55040-766c-4960-856d-3c795f85bbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:49.968622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=62 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:23:49.982333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-11b55040-766c-4960-856d-3c795f85bbc0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:49.983549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-11b55040-766c-4960-856d-3c795f85bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:49.991473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:50.262324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e4056-3b4c-4d3f-91a0-966ceeb46d75 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:50.264029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-463e4056-3b4c-4d3f-91a0-966ceeb46d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:50.321331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e4056-3b4c-4d3f-91a0-966ceeb46d75 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:50.326738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dd4e2-74eb-422c-88fb-926ec15eb6ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:50.327918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd4e2-74eb-422c-88fb-926ec15eb6ff service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:50.328414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd4e2-74eb-422c-88fb-926ec15eb6ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:50.328917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd4e2-74eb-422c-88fb-926ec15eb6ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:50.337890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dd4e2-74eb-422c-88fb-926ec15eb6ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:50.340183 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dd4e2-74eb-422c-88fb-926ec15eb6ff service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:50.349485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 760/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:23:52.585780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fdf26-d0a8-4da8-91d1-3c2c14a16c1d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:52.586341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fdf26-d0a8-4da8-91d1-3c2c14a16c1d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:52.586341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fdf26-d0a8-4da8-91d1-3c2c14a16c1d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:52.587760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 17:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:23:52.593058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80308c40-99d0-43e2-bbc0-e683ce0e264a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:52.593554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80308c40-99d0-43e2-bbc0-e683ce0e264a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:52.593691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80308c40-99d0-43e2-bbc0-e683ce0e264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:52.611715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-80308c40-99d0-43e2-bbc0-e683ce0e264a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:23:52.612887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-80308c40-99d0-43e2-bbc0-e683ce0e264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:52.621298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 17:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:23:52.892152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fdf26-d0a8-4da8-91d1-3c2c14a16c1d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:23:52.893817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-047fdf26-d0a8-4da8-91d1-3c2c14a16c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:23:52.907132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fdf26-d0a8-4da8-91d1-3c2c14a16c1d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:23:52.910642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b23f2-b560-4cee-be96-f29ae9eb9382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:23:52.911272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b23f2-b560-4cee-be96-f29ae9eb9382 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:23:52.911583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b23f2-b560-4cee-be96-f29ae9eb9382 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:23:52.911861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b23f2-b560-4cee-be96-f29ae9eb9382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:23:52.921364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b23f2-b560-4cee-be96-f29ae9eb9382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:23:52.922786 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b23f2-b560-4cee-be96-f29ae9eb9382 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:23:52.932072 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 778/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:24:02.606720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=32 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:24:02.928053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=8 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:22.767743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19792e16-d8ea-47c7-a107-e584494c7c54 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:22.768402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19792e16-d8ea-47c7-a107-e584494c7c54 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:22.768402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19792e16-d8ea-47c7-a107-e584494c7c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:22.795958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-19792e16-d8ea-47c7-a107-e584494c7c54 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:22.797242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-19792e16-d8ea-47c7-a107-e584494c7c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:22.809144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 762/1540] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:22.822135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de399b-97ea-440e-aba5-a9f18fe9a9b4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:22.823882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de399b-97ea-440e-aba5-a9f18fe9a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:22.824661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de399b-97ea-440e-aba5-a9f18fe9a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:22.854184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-89de399b-97ea-440e-aba5-a9f18fe9a9b4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:22.855992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-89de399b-97ea-440e-aba5-a9f18fe9a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:22.870943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 779/1541] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:22.875904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19792e16-d8ea-47c7-a107-e584494c7c54 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:22.893792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19792e16-d8ea-47c7-a107-e584494c7c54 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:22.898830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:22.899632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:22.899982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:22.900284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:22.901028 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:22.901381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:22.903244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:22.923229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 763/1542] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:22.931975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de399b-97ea-440e-aba5-a9f18fe9a9b4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:22.944803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de399b-97ea-440e-aba5-a9f18fe9a9b4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:22.951863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb0727-a6a2-4e40-aded-4430904fd5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:22.952934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0727-a6a2-4e40-aded-4430904fd5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:22.953453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0727-a6a2-4e40-aded-4430904fd5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:22.953944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0727-a6a2-4e40-aded-4430904fd5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:22.957060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb0727-a6a2-4e40-aded-4430904fd5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:22.959397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb0727-a6a2-4e40-aded-4430904fd5b3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.003862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 780/1543] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:22 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:23.008338 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.023772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97264-c99e-4fbf-8cef-7358f34f69c9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.024007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404ddcc-51b9-43fa-aa77-2afb1d1a5e6b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.024587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97264-c99e-4fbf-8cef-7358f34f69c9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.025096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97264-c99e-4fbf-8cef-7358f34f69c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.064330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-95e97264-c99e-4fbf-8cef-7358f34f69c9 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.065967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-95e97264-c99e-4fbf-8cef-7358f34f69c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.130067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd35ad-3823-40e7-bd68-9004ca7a0b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.130943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd35ad-3823-40e7-bd68-9004ca7a0b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.131232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd35ad-3823-40e7-bd68-9004ca7a0b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.131536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd35ad-3823-40e7-bd68-9004ca7a0b7f admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.132305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd35ad-3823-40e7-bd68-9004ca7a0b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.134602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd35ad-3823-40e7-bd68-9004ca7a0b7f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.174499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 764/1544] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:23.178754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd35ad-3823-40e7-bd68-9004ca7a0b7f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.188012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 781/1545] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:23.193716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd35ad-3823-40e7-bd68-9004ca7a0b7f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.199857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803607a-d953-4e77-afc7-de12158813e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.202417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803607a-d953-4e77-afc7-de12158813e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.204996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803607a-d953-4e77-afc7-de12158813e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.205313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803607a-d953-4e77-afc7-de12158813e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.206028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b408c-4d08-418e-bcc1-14cf5547fdd6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.206164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803607a-d953-4e77-afc7-de12158813e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.206653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b408c-4d08-418e-bcc1-14cf5547fdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.207152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b408c-4d08-418e-bcc1-14cf5547fdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.207735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803607a-d953-4e77-afc7-de12158813e9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.225084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 765/1546] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:25:23.231054 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b8f9-db0b-49db-b2ab-685fcc020b44 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.231416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b8f9-db0b-49db-b2ab-685fcc020b44 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.231641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b8f9-db0b-49db-b2ab-685fcc020b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.251443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0a3b408c-4d08-418e-bcc1-14cf5547fdd6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.253126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0a3b408c-4d08-418e-bcc1-14cf5547fdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.262303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-74d6b8f9-db0b-49db-b2ab-685fcc020b44 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.263666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-74d6b8f9-db0b-49db-b2ab-685fcc020b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.280799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 766/1547] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.281626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 782/1548] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:23.281780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6b8f9-db0b-49db-b2ab-685fcc020b44 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.283542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647a380-6678-4fff-a126-d925b54844e6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.284060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647a380-6678-4fff-a126-d925b54844e6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.284546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647a380-6678-4fff-a126-d925b54844e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.296044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6b8f9-db0b-49db-b2ab-685fcc020b44 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.301052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9be7d3-1b73-4739-9fdc-d7083d10b1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.301731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9be7d3-1b73-4739-9fdc-d7083d10b1ca admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.302016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9be7d3-1b73-4739-9fdc-d7083d10b1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.302292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9be7d3-1b73-4739-9fdc-d7083d10b1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.303030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9be7d3-1b73-4739-9fdc-d7083d10b1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.305864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9be7d3-1b73-4739-9fdc-d7083d10b1ca admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.318960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 767/1549] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:23.320278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9be7d3-1b73-4739-9fdc-d7083d10b1ca admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.326666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9647a380-6678-4fff-a126-d925b54844e6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.328349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9647a380-6678-4fff-a126-d925b54844e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.333826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9be7d3-1b73-4739-9fdc-d7083d10b1ca admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.338180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 783/1550] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.339498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc1bdc-d8b5-4d35-b477-5e329a76c7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.340233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1bdc-d8b5-4d35-b477-5e329a76c7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.340520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1bdc-d8b5-4d35-b477-5e329a76c7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.340804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1bdc-d8b5-4d35-b477-5e329a76c7c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.341431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f3500-0b60-4fa3-8f43-b59b0033ff10 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.341729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc1bdc-d8b5-4d35-b477-5e329a76c7c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:23.342055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc1bdc-d8b5-4d35-b477-5e329a76c7c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.342975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f3500-0b60-4fa3-8f43-b59b0033ff10 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.343641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f3500-0b60-4fa3-8f43-b59b0033ff10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.343767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc1bdc-d8b5-4d35-b477-5e329a76c7c2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.361291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 768/1551] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:23.367362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2347f-6ad7-4118-b1a3-4d383ad31be2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.367362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2347f-6ad7-4118-b1a3-4d383ad31be2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.367362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2347f-6ad7-4118-b1a3-4d383ad31be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.371362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e50f3500-0b60-4fa3-8f43-b59b0033ff10 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.372317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e50f3500-0b60-4fa3-8f43-b59b0033ff10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.384678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 784/1552] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.386021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf147e1d-d3d2-4eb0-bfeb-9280dda40920 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.386308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf147e1d-d3d2-4eb0-bfeb-9280dda40920 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.386584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf147e1d-d3d2-4eb0-bfeb-9280dda40920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.397445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-08d2347f-6ad7-4118-b1a3-4d383ad31be2 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.398661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-08d2347f-6ad7-4118-b1a3-4d383ad31be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.406820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bf147e1d-d3d2-4eb0-bfeb-9280dda40920 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.408087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bf147e1d-d3d2-4eb0-bfeb-9280dda40920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.408631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 769/1553] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.409870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2347f-6ad7-4118-b1a3-4d383ad31be2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.418992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 785/1554] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.418992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094ff0-7ee5-4c5d-9323-a84163936fc6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.418992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094ff0-7ee5-4c5d-9323-a84163936fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.419458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094ff0-7ee5-4c5d-9323-a84163936fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.427249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2347f-6ad7-4118-b1a3-4d383ad31be2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.432401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.434233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.434535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.434814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] PATH_INFO: `/v3/projects/1a433429eb484bd4b1dae47092f38c2a/users/e21616bf7b5c44e6ad6a8c4a9cc6d8af/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.438458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3d094ff0-7ee5-4c5d-9323-a84163936fc6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.439698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3d094ff0-7ee5-4c5d-9323-a84163936fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.447714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 786/1555] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.452141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d094ff0-7ee5-4c5d-9323-a84163936fc6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.452283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a433429eb484bd4b1dae47092f38c2a, user_id=e21616bf7b5c44e6ad6a8c4a9cc6d8af, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.455383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:25:23.455383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 28 17:25:23.455383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: May 28 17:25:23.455773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.458596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d094ff0-7ee5-4c5d-9323-a84163936fc6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.464719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063f4e1-e644-4773-911a-6a437d2a80f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.465481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063f4e1-e644-4773-911a-6a437d2a80f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.465851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063f4e1-e644-4773-911a-6a437d2a80f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.466138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063f4e1-e644-4773-911a-6a437d2a80f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.466875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063f4e1-e644-4773-911a-6a437d2a80f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.468885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063f4e1-e644-4773-911a-6a437d2a80f0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.470001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 770/1556] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:23 2022] PUT /identity/v3/projects/1a433429eb484bd4b1dae47092f38c2a/users/e21616bf7b5c44e6ad6a8c4a9cc6d8af/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:23.470813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.496359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 787/1557] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:23.498008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75968b7c-80a4-417d-9f31-86d1d205a1a4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.498299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75968b7c-80a4-417d-9f31-86d1d205a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.498583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75968b7c-80a4-417d-9f31-86d1d205a1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.518407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-75968b7c-80a4-417d-9f31-86d1d205a1a4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.521153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-75968b7c-80a4-417d-9f31-86d1d205a1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.531242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:23.535074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9927dd-e019-49c8-85be-33e41bfeb301 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.582875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 788/1558] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.584312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f30b8-91b0-428c-abf5-cba0ac904147 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.584615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f30b8-91b0-428c-abf5-cba0ac904147 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.584897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f30b8-91b0-428c-abf5-cba0ac904147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.601129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.601803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.602087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.602394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.603123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:23.603466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ac6f30b8-91b0-428c-abf5-cba0ac904147 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.603588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.605394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ac6f30b8-91b0-428c-abf5-cba0ac904147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.605707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.613284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 789/1559] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.614502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b672a7-a4bd-40a6-85c2-b618155a66da None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.614787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b672a7-a4bd-40a6-85c2-b618155a66da None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.615124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b672a7-a4bd-40a6-85c2-b618155a66da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.617606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 771/1560] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:23.618372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.628292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6bf5d-783c-4616-8808-8609303add64 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.634117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977d14d-e964-4fe9-9875-bbc0489c25a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.634836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977d14d-e964-4fe9-9875-bbc0489c25a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.635218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977d14d-e964-4fe9-9875-bbc0489c25a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.635503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977d14d-e964-4fe9-9875-bbc0489c25a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.636331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977d14d-e964-4fe9-9875-bbc0489c25a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:23.636644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977d14d-e964-4fe9-9875-bbc0489c25a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.637216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f1b672a7-a4bd-40a6-85c2-b618155a66da None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.638396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977d14d-e964-4fe9-9875-bbc0489c25a9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.638540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f1b672a7-a4bd-40a6-85c2-b618155a66da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.647039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 790/1561] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.648311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e812-0ded-47c4-ad77-d2e1dc1e42f3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.648603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e812-0ded-47c4-ad77-d2e1dc1e42f3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.648883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e812-0ded-47c4-ad77-d2e1dc1e42f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.661672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 772/1562] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:23.661672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef8554-d762-4a95-adb1-b5dbc35cb697 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.661672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef8554-d762-4a95-adb1-b5dbc35cb697 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.661672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef8554-d762-4a95-adb1-b5dbc35cb697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.664414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:23.676370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ecef8554-d762-4a95-adb1-b5dbc35cb697 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.676673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a301e812-0ded-47c4-ad77-d2e1dc1e42f3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.677588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ecef8554-d762-4a95-adb1-b5dbc35cb697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.677845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a301e812-0ded-47c4-ad77-d2e1dc1e42f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.685637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 773/1563] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.686474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef8554-d762-4a95-adb1-b5dbc35cb697 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.688427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 791/1564] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.690174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91748f0-6cca-4aff-885e-b1265a83799d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.690471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91748f0-6cca-4aff-885e-b1265a83799d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.690747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91748f0-6cca-4aff-885e-b1265a83799d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.697954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef8554-d762-4a95-adb1-b5dbc35cb697 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.703039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c8303-939b-467e-9cd0-e887c753b077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.703767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8303-939b-467e-9cd0-e887c753b077 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.704056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8303-939b-467e-9cd0-e887c753b077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.704331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8303-939b-467e-9cd0-e887c753b077 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.705020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c8303-939b-467e-9cd0-e887c753b077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.707081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c8303-939b-467e-9cd0-e887c753b077 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.713060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d91748f0-6cca-4aff-885e-b1265a83799d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.715448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d91748f0-6cca-4aff-885e-b1265a83799d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.725557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 792/1565] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.726535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91748f0-6cca-4aff-885e-b1265a83799d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.732849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 774/1566] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/users => generated 292 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:23.734450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c8303-939b-467e-9cd0-e887c753b077 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.737438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91748f0-6cca-4aff-885e-b1265a83799d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.742109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.742770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.743048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.743324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.744343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:23.744484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c8303-939b-467e-9cd0-e887c753b077 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.744653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.745745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 17:25:23.745745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: warnings.warn(msg) May 28 17:25:23.745745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: May 28 17:25:23.746616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.749391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.749611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.749893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.750169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.750856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.751893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:25:23.751893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 28 17:25:23.751893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: May 28 17:25:23.753115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.758302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 793/1567] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:23.759162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.769995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dc0f5-fd7e-47aa-acac-9048e2f409d1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.775568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcd353-fbd8-4770-812e-f19ada662785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.776831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcd353-fbd8-4770-812e-f19ada662785 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.777107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcd353-fbd8-4770-812e-f19ada662785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.777390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcd353-fbd8-4770-812e-f19ada662785 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.778116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcd353-fbd8-4770-812e-f19ada662785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:23.778431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcd353-fbd8-4770-812e-f19ada662785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.780053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcd353-fbd8-4770-812e-f19ada662785 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.784372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 775/1568] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:23.784372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.792801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 794/1569] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:23.794125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1eedaf-f0fb-4e73-8b00-19f32174b8fe None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.794411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1eedaf-f0fb-4e73-8b00-19f32174b8fe None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.794688 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1eedaf-f0fb-4e73-8b00-19f32174b8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.812399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5a1eedaf-f0fb-4e73-8b00-19f32174b8fe None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:23.813593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5a1eedaf-f0fb-4e73-8b00-19f32174b8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:23.852177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bb86-7909-4883-bf63-3bf92673644a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.856996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049ea7f-21a8-40e4-98f5-9a26c33bb374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.857662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049ea7f-21a8-40e4-98f5-9a26c33bb374 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.857949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049ea7f-21a8-40e4-98f5-9a26c33bb374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.858224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049ea7f-21a8-40e4-98f5-9a26c33bb374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.859119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049ea7f-21a8-40e4-98f5-9a26c33bb374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.860848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049ea7f-21a8-40e4-98f5-9a26c33bb374 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.873689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 795/1570] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:23.875229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1eedaf-f0fb-4e73-8b00-19f32174b8fe None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.887495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 776/1571] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:23 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:23.888345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049ea7f-21a8-40e4-98f5-9a26c33bb374 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.915883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:23.933672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1eedaf-f0fb-4e73-8b00-19f32174b8fe None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.952689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049ea7f-21a8-40e4-98f5-9a26c33bb374 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.957630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.957963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.958246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.958524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.959249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:23.959587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.961200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.972661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 777/1572] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:23.973595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:23.982933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854970ad-6763-438b-8b8b-8f85c8b0e08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.983689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854970ad-6763-438b-8b8b-8f85c8b0e08a service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.984003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854970ad-6763-438b-8b8b-8f85c8b0e08a service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.984286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854970ad-6763-438b-8b8b-8f85c8b0e08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.989726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461c59-2889-4d7b-8dcc-133b4899cb85 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:23.994032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854970ad-6763-438b-8b8b-8f85c8b0e08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.994494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7d837-51c8-47e7-b30f-253c51baef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:23.995715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7d837-51c8-47e7-b30f-253c51baef5a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:23.996000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7d837-51c8-47e7-b30f-253c51baef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:23.996120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854970ad-6763-438b-8b8b-8f85c8b0e08a service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:23.996279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7d837-51c8-47e7-b30f-253c51baef5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:23.997000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7d837-51c8-47e7-b30f-253c51baef5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:23.998574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7d837-51c8-47e7-b30f-253c51baef5a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.005861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 796/1573] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:24.006652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854970ad-6763-438b-8b8b-8f85c8b0e08a service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.011696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 778/1574] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.011696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7d837-51c8-47e7-b30f-253c51baef5a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.016557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854970ad-6763-438b-8b8b-8f85c8b0e08a service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.018923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7d837-51c8-47e7-b30f-253c51baef5a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.021649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308109d0-dfa7-4319-ba07-beefaf201beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.022340 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308109d0-dfa7-4319-ba07-beefaf201beb admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.022703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308109d0-dfa7-4319-ba07-beefaf201beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.023054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308109d0-dfa7-4319-ba07-beefaf201beb admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.023849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308109d0-dfa7-4319-ba07-beefaf201beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.024179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699d50-c3c8-4721-8246-32df68ae15f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.024848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699d50-c3c8-4721-8246-32df68ae15f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.025133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699d50-c3c8-4721-8246-32df68ae15f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.025405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699d50-c3c8-4721-8246-32df68ae15f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.025880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308109d0-dfa7-4319-ba07-beefaf201beb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.026225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699d50-c3c8-4721-8246-32df68ae15f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.028178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699d50-c3c8-4721-8246-32df68ae15f5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.054820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 779/1575] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.054820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699d50-c3c8-4721-8246-32df68ae15f5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.060414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 797/1576] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.060414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308109d0-dfa7-4319-ba07-beefaf201beb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.063063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:24.066022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699d50-c3c8-4721-8246-32df68ae15f5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.136535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ae42-aaaa-402e-8bf5-ec15d2852650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.137168 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ae42-aaaa-402e-8bf5-ec15d2852650 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.137449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ae42-aaaa-402e-8bf5-ec15d2852650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.137727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ae42-aaaa-402e-8bf5-ec15d2852650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.138439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055ae42-aaaa-402e-8bf5-ec15d2852650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.140424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055ae42-aaaa-402e-8bf5-ec15d2852650 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.146510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308109d0-dfa7-4319-ba07-beefaf201beb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.153446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.154141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.154423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.154698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.155484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:24.155797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.157682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.172575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 780/1577] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.172575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ae42-aaaa-402e-8bf5-ec15d2852650 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.175453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 798/1578] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:24.180271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.193044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f04ef-3571-4043-8382-5961c16f5281 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.248184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ae42-aaaa-402e-8bf5-ec15d2852650 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.265760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6c1c8-2804-4a9b-a69c-e9797387c3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.267087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6c1c8-2804-4a9b-a69c-e9797387c3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.267467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6c1c8-2804-4a9b-a69c-e9797387c3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.267801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6c1c8-2804-4a9b-a69c-e9797387c3e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.268510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6c1c8-2804-4a9b-a69c-e9797387c3e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.270857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6c1c8-2804-4a9b-a69c-e9797387c3e0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.271593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20b4df-c90c-4eb0-8d1e-f18b14e8acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.272576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b4df-c90c-4eb0-8d1e-f18b14e8acdd admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.272798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b4df-c90c-4eb0-8d1e-f18b14e8acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.273077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b4df-c90c-4eb0-8d1e-f18b14e8acdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.273776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20b4df-c90c-4eb0-8d1e-f18b14e8acdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.275854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20b4df-c90c-4eb0-8d1e-f18b14e8acdd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.294954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 781/1579] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.296047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6c1c8-2804-4a9b-a69c-e9797387c3e0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.303257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:24.305785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 799/1580] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.306721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20b4df-c90c-4eb0-8d1e-f18b14e8acdd admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.312064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6c1c8-2804-4a9b-a69c-e9797387c3e0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.378400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784dbcc-54c7-48ea-bfac-b55e0788caeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.379081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784dbcc-54c7-48ea-bfac-b55e0788caeb admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.379388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784dbcc-54c7-48ea-bfac-b55e0788caeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.379668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784dbcc-54c7-48ea-bfac-b55e0788caeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.380467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784dbcc-54c7-48ea-bfac-b55e0788caeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.382111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784dbcc-54c7-48ea-bfac-b55e0788caeb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.386505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20b4df-c90c-4eb0-8d1e-f18b14e8acdd admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.392043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98cea8-fe5a-4477-a078-9a1e46fa9930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.392882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 782/1581] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.393212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98cea8-fe5a-4477-a078-9a1e46fa9930 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.393678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784dbcc-54c7-48ea-bfac-b55e0788caeb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.393932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98cea8-fe5a-4477-a078-9a1e46fa9930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.394424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98cea8-fe5a-4477-a078-9a1e46fa9930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.395229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98cea8-fe5a-4477-a078-9a1e46fa9930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.396957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98cea8-fe5a-4477-a078-9a1e46fa9930 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.406467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784dbcc-54c7-48ea-bfac-b55e0788caeb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.411835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2e04-5ce7-4853-a005-1f40707a5ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.412423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2e04-5ce7-4853-a005-1f40707a5ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.412988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2e04-5ce7-4853-a005-1f40707a5ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.413524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2e04-5ce7-4853-a005-1f40707a5ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.414293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af2e04-5ce7-4853-a005-1f40707a5ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.415409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 800/1582] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.416054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98cea8-fe5a-4477-a078-9a1e46fa9930 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.420092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af2e04-5ce7-4853-a005-1f40707a5ad5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.431904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98cea8-fe5a-4477-a078-9a1e46fa9930 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.438465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.439424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.439782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.440195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.440962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.442200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:25:24.442200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: warnings.warn(msg) May 28 17:25:24.442200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: May 28 17:25:24.443650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.447987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 783/1583] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/users => generated 302 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.449394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2e04-5ce7-4853-a005-1f40707a5ad5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.461819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2e04-5ce7-4853-a005-1f40707a5ad5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.465981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:24.469672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207f14a-c207-4cf3-88f2-7cade585ec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.470607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207f14a-c207-4cf3-88f2-7cade585ec1a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.471053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207f14a-c207-4cf3-88f2-7cade585ec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.471522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207f14a-c207-4cf3-88f2-7cade585ec1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.472402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207f14a-c207-4cf3-88f2-7cade585ec1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.474904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207f14a-c207-4cf3-88f2-7cade585ec1a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.480532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 801/1584] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/users => generated 289 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.481447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.496022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95046d2a-540f-4832-b533-31b6e624882c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.503824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45756785-f3c9-449f-8581-c73a023cebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.505045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45756785-f3c9-449f-8581-c73a023cebb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.505045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45756785-f3c9-449f-8581-c73a023cebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.505368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45756785-f3c9-449f-8581-c73a023cebb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.506110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45756785-f3c9-449f-8581-c73a023cebb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.508197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45756785-f3c9-449f-8581-c73a023cebb4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.512690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 784/1585] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.512690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207f14a-c207-4cf3-88f2-7cade585ec1a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.525390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 802/1586] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.526783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45756785-f3c9-449f-8581-c73a023cebb4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.537331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45756785-f3c9-449f-8581-c73a023cebb4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.587435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207f14a-c207-4cf3-88f2-7cade585ec1a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.593351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0d30a-969d-4125-80c3-018b60d3aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.594037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0d30a-969d-4125-80c3-018b60d3aaec admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.594327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0d30a-969d-4125-80c3-018b60d3aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.594604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0d30a-969d-4125-80c3-018b60d3aaec admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.595367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0d30a-969d-4125-80c3-018b60d3aaec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.597512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0d30a-969d-4125-80c3-018b60d3aaec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.601649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb355a5-31e6-410f-bf4a-f78cd269b6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.602835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb355a5-31e6-410f-bf4a-f78cd269b6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.603440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb355a5-31e6-410f-bf4a-f78cd269b6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.603909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb355a5-31e6-410f-bf4a-f78cd269b6c7 admin admin] PATH_INFO: `/v3/projects/2798e3c92658428f951e104bc6458744/users/55f494e6fed84b7485541ef5cd668701/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.623720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 785/1587] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.625805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb355a5-31e6-410f-bf4a-f78cd269b6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2798e3c92658428f951e104bc6458744, user_id=55f494e6fed84b7485541ef5cd668701, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.627362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0d30a-969d-4125-80c3-018b60d3aaec admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.628874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb355a5-31e6-410f-bf4a-f78cd269b6c7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.637498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0d30a-969d-4125-80c3-018b60d3aaec admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.654773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 803/1588] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:24 2022] PUT /identity/v3/projects/2798e3c92658428f951e104bc6458744/users/55f494e6fed84b7485541ef5cd668701/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:24.654773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb355a5-31e6-410f-bf4a-f78cd269b6c7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.696107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:24.709703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccec91-7632-4647-aeb3-f382cc27b90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.710753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccec91-7632-4647-aeb3-f382cc27b90a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.711019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccec91-7632-4647-aeb3-f382cc27b90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.711430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccec91-7632-4647-aeb3-f382cc27b90a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.712632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccec91-7632-4647-aeb3-f382cc27b90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.714478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccec91-7632-4647-aeb3-f382cc27b90a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.727324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 786/1589] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.728369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccec91-7632-4647-aeb3-f382cc27b90a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.740737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccec91-7632-4647-aeb3-f382cc27b90a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.752508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb355a5-31e6-410f-bf4a-f78cd269b6c7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.755004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2627a-0050-46a0-a6f0-1d96ba798c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.755738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2627a-0050-46a0-a6f0-1d96ba798c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.756021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2627a-0050-46a0-a6f0-1d96ba798c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.756298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2627a-0050-46a0-a6f0-1d96ba798c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.757052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2627a-0050-46a0-a6f0-1d96ba798c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.759120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2627a-0050-46a0-a6f0-1d96ba798c55 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.759423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756e0c6-9b48-4731-a847-9f47b9081f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.760246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756e0c6-9b48-4731-a847-9f47b9081f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.760728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756e0c6-9b48-4731-a847-9f47b9081f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.761167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756e0c6-9b48-4731-a847-9f47b9081f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.762074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756e0c6-9b48-4731-a847-9f47b9081f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.764524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756e0c6-9b48-4731-a847-9f47b9081f14 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.778029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 787/1590] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.778153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 804/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.778153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756e0c6-9b48-4731-a847-9f47b9081f14 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.778301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2627a-0050-46a0-a6f0-1d96ba798c55 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.786581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2627a-0050-46a0-a6f0-1d96ba798c55 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.792460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec2ed-37dc-4573-90dd-4f083e6cfcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.793218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec2ed-37dc-4573-90dd-4f083e6cfcc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.793491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec2ed-37dc-4573-90dd-4f083e6cfcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.793769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec2ed-37dc-4573-90dd-4f083e6cfcc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.794464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ec2ed-37dc-4573-90dd-4f083e6cfcc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.796455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ec2ed-37dc-4573-90dd-4f083e6cfcc5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.798551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756e0c6-9b48-4731-a847-9f47b9081f14 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.804208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccc298-8e64-4b7e-982a-58736bc542a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.804880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccc298-8e64-4b7e-982a-58736bc542a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.805286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccc298-8e64-4b7e-982a-58736bc542a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.805571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccc298-8e64-4b7e-982a-58736bc542a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.806532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccc298-8e64-4b7e-982a-58736bc542a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.808999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccc298-8e64-4b7e-982a-58736bc542a9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.817366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 788/1592] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:24.818497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec2ed-37dc-4573-90dd-4f083e6cfcc5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.828000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 805/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.829771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55219-db32-4cd0-b3db-ded8141f7d17 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.830066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec2ed-37dc-4573-90dd-4f083e6cfcc5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.830581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55219-db32-4cd0-b3db-ded8141f7d17 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.831035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55219-db32-4cd0-b3db-ded8141f7d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.834975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcaae52-17e8-4523-9308-b69783f77d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.835670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaae52-17e8-4523-9308-b69783f77d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.835954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaae52-17e8-4523-9308-b69783f77d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.836231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaae52-17e8-4523-9308-b69783f77d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.836967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcaae52-17e8-4523-9308-b69783f77d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:24.838564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcaae52-17e8-4523-9308-b69783f77d1e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:24.862365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-31e55219-db32-4cd0-b3db-ded8141f7d17 None None] MFA Rules not processed for user `55f494e6fed84b7485541ef5cd668701`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:24.863931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-31e55219-db32-4cd0-b3db-ded8141f7d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:24.922912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:24.950270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 806/1594] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:24.951127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e55219-db32-4cd0-b3db-ded8141f7d17 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:24.968945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e55219-db32-4cd0-b3db-ded8141f7d17 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:24.974426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa5bbc-abeb-4e24-8f5e-c2a592fad50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:24.975447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa5bbc-abeb-4e24-8f5e-c2a592fad50c admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.975796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa5bbc-abeb-4e24-8f5e-c2a592fad50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.976362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa5bbc-abeb-4e24-8f5e-c2a592fad50c admin admin] PATH_INFO: `/v3/projects/29920bb776f840388b047e19dbac990f/users/8c0632580b96498c9b010a80e9e0f659/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:24.981329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 789/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:24.981329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4436ff-1ac3-48e1-81a1-8ac9699f4481 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:24.981329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4436ff-1ac3-48e1-81a1-8ac9699f4481 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:24.981329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4436ff-1ac3-48e1-81a1-8ac9699f4481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.002452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa5bbc-abeb-4e24-8f5e-c2a592fad50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=29920bb776f840388b047e19dbac990f, user_id=8c0632580b96498c9b010a80e9e0f659, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.005199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa5bbc-abeb-4e24-8f5e-c2a592fad50c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.009843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ea4436ff-1ac3-48e1-81a1-8ac9699f4481 None None] MFA Rules not processed for user `e21616bf7b5c44e6ad6a8c4a9cc6d8af`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.012069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ea4436ff-1ac3-48e1-81a1-8ac9699f4481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.019455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 807/1596] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:24 2022] PUT /identity/v3/projects/29920bb776f840388b047e19dbac990f/users/8c0632580b96498c9b010a80e9e0f659/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:25.020004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa5bbc-abeb-4e24-8f5e-c2a592fad50c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.076979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 790/1597] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:25.077940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4436ff-1ac3-48e1-81a1-8ac9699f4481 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.084754 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa5bbc-abeb-4e24-8f5e-c2a592fad50c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.091120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69973947-f2b3-4541-bba4-8191f4851805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.091825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973947-f2b3-4541-bba4-8191f4851805 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.092105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973947-f2b3-4541-bba4-8191f4851805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.092401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973947-f2b3-4541-bba4-8191f4851805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.093138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69973947-f2b3-4541-bba4-8191f4851805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.094732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69973947-f2b3-4541-bba4-8191f4851805 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.095018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4436ff-1ac3-48e1-81a1-8ac9699f4481 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.099584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bacb-ba39-4766-add4-8163d0b2fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.100246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bacb-ba39-4766-add4-8163d0b2fa89 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.100530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bacb-ba39-4766-add4-8163d0b2fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.100809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bacb-ba39-4766-add4-8163d0b2fa89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.101536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3bacb-ba39-4766-add4-8163d0b2fa89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.103100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3bacb-ba39-4766-add4-8163d0b2fa89 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.105091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 808/1598] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:25.105895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69973947-f2b3-4541-bba4-8191f4851805 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.117853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 791/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:25.117853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bacb-ba39-4766-add4-8163d0b2fa89 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.118151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69973947-f2b3-4541-bba4-8191f4851805 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.123554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5e9c-145a-4b16-aeb0-a6604b8ca532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.124488 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5e9c-145a-4b16-aeb0-a6604b8ca532 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.124988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5e9c-145a-4b16-aeb0-a6604b8ca532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.125434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5e9c-145a-4b16-aeb0-a6604b8ca532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.126234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa5e9c-145a-4b16-aeb0-a6604b8ca532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.128130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa5e9c-145a-4b16-aeb0-a6604b8ca532 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.132758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bacb-ba39-4766-add4-8163d0b2fa89 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.138537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de914ab-b540-4d07-badb-f32a03af06c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.139290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 809/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:25.139451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de914ab-b540-4d07-badb-f32a03af06c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.139746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de914ab-b540-4d07-badb-f32a03af06c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.140012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de914ab-b540-4d07-badb-f32a03af06c1 admin admin] PATH_INFO: `/v3/projects/24240be31473431cb8690f8cb08a3cd5/users/3fd43777943042058151ec9ae8479155/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.142719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5e9c-145a-4b16-aeb0-a6604b8ca532 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.152728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5e9c-145a-4b16-aeb0-a6604b8ca532 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.156181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de914ab-b540-4d07-badb-f32a03af06c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24240be31473431cb8690f8cb08a3cd5, user_id=3fd43777943042058151ec9ae8479155, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.157255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672fe1-44d2-46b5-81f8-53074bee5ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.157900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672fe1-44d2-46b5-81f8-53074bee5ada admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.158179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672fe1-44d2-46b5-81f8-53074bee5ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.158449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672fe1-44d2-46b5-81f8-53074bee5ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.158799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de914ab-b540-4d07-badb-f32a03af06c1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.159161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55672fe1-44d2-46b5-81f8-53074bee5ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.160763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55672fe1-44d2-46b5-81f8-53074bee5ada admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.172493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 810/1601] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:25.182664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 792/1602] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:25 2022] PUT /identity/v3/projects/24240be31473431cb8690f8cb08a3cd5/users/3fd43777943042058151ec9ae8479155/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:25.182866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63eac1-fea9-4c05-a42c-272ccc0cc11d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.182866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63eac1-fea9-4c05-a42c-272ccc0cc11d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.183146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fca59-c140-46f1-bc1d-88ccd80853bd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.183250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63eac1-fea9-4c05-a42c-272ccc0cc11d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.183372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fca59-c140-46f1-bc1d-88ccd80853bd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.183372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fca59-c140-46f1-bc1d-88ccd80853bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.183566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 811/1603] 10.222.0.33 () {54 vars in 961 bytes} [Sat May 28 17:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:25:25.183566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d993b9f-10be-4146-8120-cf729bac7170 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.183566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d993b9f-10be-4146-8120-cf729bac7170 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.183566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d993b9f-10be-4146-8120-cf729bac7170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.198664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6d993b9f-10be-4146-8120-cf729bac7170 None None] MFA Rules not processed for user `e21616bf7b5c44e6ad6a8c4a9cc6d8af`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.199984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6d993b9f-10be-4146-8120-cf729bac7170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.213159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2a7fca59-c140-46f1-bc1d-88ccd80853bd None None] MFA Rules not processed for user `8c0632580b96498c9b010a80e9e0f659`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.215495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2a7fca59-c140-46f1-bc1d-88ccd80853bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.258530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 812/1604] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:25.259156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d993b9f-10be-4146-8120-cf729bac7170 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.267960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:25.288940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 793/1605] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:25.289584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fca59-c140-46f1-bc1d-88ccd80853bd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.319845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:25.341911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d993b9f-10be-4146-8120-cf729bac7170 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.346459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a837c95-1cf5-4709-b4e0-dc93bcbdfc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.347357 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837c95-1cf5-4709-b4e0-dc93bcbdfc34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.347641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837c95-1cf5-4709-b4e0-dc93bcbdfc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.347908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837c95-1cf5-4709-b4e0-dc93bcbdfc34 admin admin] PATH_INFO: `/v3/projects/90b378551d1c467e911cf01274c97d85/users/7fcc62869ac34381a2327be46511137b/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.354460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fca59-c140-46f1-bc1d-88ccd80853bd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.359145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02be9f6-328b-4107-abc3-e330991e4e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.360202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02be9f6-328b-4107-abc3-e330991e4e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.360569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02be9f6-328b-4107-abc3-e330991e4e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.360849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02be9f6-328b-4107-abc3-e330991e4e1d admin admin] PATH_INFO: `/v3/projects/eeb3e12d39cb49e3bbac2c7d030f8995/users/93efc8c9018e4b3496b127a410a3aecb/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.362253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a837c95-1cf5-4709-b4e0-dc93bcbdfc34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b378551d1c467e911cf01274c97d85, user_id=7fcc62869ac34381a2327be46511137b, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.365001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a837c95-1cf5-4709-b4e0-dc93bcbdfc34 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.376139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02be9f6-328b-4107-abc3-e330991e4e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb3e12d39cb49e3bbac2c7d030f8995, user_id=93efc8c9018e4b3496b127a410a3aecb, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.378823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02be9f6-328b-4107-abc3-e330991e4e1d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.379483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 813/1606] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:25 2022] PUT /identity/v3/projects/90b378551d1c467e911cf01274c97d85/users/7fcc62869ac34381a2327be46511137b/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:25.380697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a837c95-1cf5-4709-b4e0-dc93bcbdfc34 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.393097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 794/1607] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:25 2022] PUT /identity/v3/projects/eeb3e12d39cb49e3bbac2c7d030f8995/users/93efc8c9018e4b3496b127a410a3aecb/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:25.393895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02be9f6-328b-4107-abc3-e330991e4e1d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.452628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a837c95-1cf5-4709-b4e0-dc93bcbdfc34 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.463699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02be9f6-328b-4107-abc3-e330991e4e1d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.469148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1881-86c9-4839-8bc0-5c409483f680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.470036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1881-86c9-4839-8bc0-5c409483f680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.470314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1881-86c9-4839-8bc0-5c409483f680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.470585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1881-86c9-4839-8bc0-5c409483f680 admin admin] PATH_INFO: `/v3/projects/6d6dc24eb1bb42c7a348f24ea101b166/users/232a1bcc0b9644abb625e284e117212c/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.485102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc1881-86c9-4839-8bc0-5c409483f680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6dc24eb1bb42c7a348f24ea101b166, user_id=232a1bcc0b9644abb625e284e117212c, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.487801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc1881-86c9-4839-8bc0-5c409483f680 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.505125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 795/1608] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:25 2022] PUT /identity/v3/projects/6d6dc24eb1bb42c7a348f24ea101b166/users/232a1bcc0b9644abb625e284e117212c/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:25.506393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6ec93-5fd7-4570-8fb6-cacdce50af82 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.506709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6ec93-5fd7-4570-8fb6-cacdce50af82 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.507020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6ec93-5fd7-4570-8fb6-cacdce50af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.523106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87384ee-e496-4564-bdd8-8cfa465ceca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.523814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87384ee-e496-4564-bdd8-8cfa465ceca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.524098 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87384ee-e496-4564-bdd8-8cfa465ceca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.524375 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87384ee-e496-4564-bdd8-8cfa465ceca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.525105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87384ee-e496-4564-bdd8-8cfa465ceca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.527262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-76a6ec93-5fd7-4570-8fb6-cacdce50af82 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.527426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87384ee-e496-4564-bdd8-8cfa465ceca0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.528517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-76a6ec93-5fd7-4570-8fb6-cacdce50af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.537803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 814/1609] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:25.539142 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae768e-8996-4307-be4e-5a5095d6cb20 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.539457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae768e-8996-4307-be4e-5a5095d6cb20 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.539861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae768e-8996-4307-be4e-5a5095d6cb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.566261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-66ae768e-8996-4307-be4e-5a5095d6cb20 None None] MFA Rules not processed for user `3fd43777943042058151ec9ae8479155`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.567498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-66ae768e-8996-4307-be4e-5a5095d6cb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.582215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:25.602573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 796/1610] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:25.604284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6873e-97ad-44d7-86b9-92b8775f5732 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.604781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6873e-97ad-44d7-86b9-92b8775f5732 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.605082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6873e-97ad-44d7-86b9-92b8775f5732 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.606348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 797/1611] 10.222.0.33 () {54 vars in 961 bytes} [Sat May 28 17:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:25:25.607107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6873e-97ad-44d7-86b9-92b8775f5732 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.631317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:25.637975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 815/1612] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:25.639677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f4d8d-08be-4e00-acc5-16996b1d7232 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.640010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f4d8d-08be-4e00-acc5-16996b1d7232 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.640339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f4d8d-08be-4e00-acc5-16996b1d7232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.658689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-185f4d8d-08be-4e00-acc5-16996b1d7232 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.660115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-185f4d8d-08be-4e00-acc5-16996b1d7232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.678215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6873e-97ad-44d7-86b9-92b8775f5732 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.683558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f5117-2544-4a46-8f99-8cf1db9c42f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.684215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f5117-2544-4a46-8f99-8cf1db9c42f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.684493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f5117-2544-4a46-8f99-8cf1db9c42f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.684769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f5117-2544-4a46-8f99-8cf1db9c42f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.685496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f5117-2544-4a46-8f99-8cf1db9c42f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.687438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f5117-2544-4a46-8f99-8cf1db9c42f5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.714822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 798/1613] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:25.718176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a8c24-f806-4cec-9098-dcc6006c981d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.718176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a8c24-f806-4cec-9098-dcc6006c981d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.718176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a8c24-f806-4cec-9098-dcc6006c981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.723047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 816/1614] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:25.727967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d970764-630e-4f52-8552-131fd93191ca None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.728118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d970764-630e-4f52-8552-131fd93191ca None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.728427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d970764-630e-4f52-8552-131fd93191ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.744793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c11a8c24-f806-4cec-9098-dcc6006c981d None None] MFA Rules not processed for user `93efc8c9018e4b3496b127a410a3aecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.746981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c11a8c24-f806-4cec-9098-dcc6006c981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.757634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9d970764-630e-4f52-8552-131fd93191ca None None] MFA Rules not processed for user `232a1bcc0b9644abb625e284e117212c`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.758822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9d970764-630e-4f52-8552-131fd93191ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.811224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 799/1615] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:25.812086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a8c24-f806-4cec-9098-dcc6006c981d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.826015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 817/1616] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:25.826852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d970764-630e-4f52-8552-131fd93191ca None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.876448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a8c24-f806-4cec-9098-dcc6006c981d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.881036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4232d02-a2dd-4255-8a6e-9a4e51a16642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.881923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232d02-a2dd-4255-8a6e-9a4e51a16642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.882201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232d02-a2dd-4255-8a6e-9a4e51a16642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.882473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232d02-a2dd-4255-8a6e-9a4e51a16642 admin admin] PATH_INFO: `/v3/projects/8e92a5dc387f47c99ce98da4d6fd9d92/users/927f21ef1ca74ec58f85181728933e82/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.892026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:25.899646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4232d02-a2dd-4255-8a6e-9a4e51a16642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e92a5dc387f47c99ce98da4d6fd9d92, user_id=927f21ef1ca74ec58f85181728933e82, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.903783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d970764-630e-4f52-8552-131fd93191ca None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:25.904158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4232d02-a2dd-4255-8a6e-9a4e51a16642 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.909775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2db46-a7b5-45d9-aa2a-f10131887223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:25.910644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2db46-a7b5-45d9-aa2a-f10131887223 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.911026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2db46-a7b5-45d9-aa2a-f10131887223 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.911309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2db46-a7b5-45d9-aa2a-f10131887223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.920859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 800/1617] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:25 2022] PUT /identity/v3/projects/8e92a5dc387f47c99ce98da4d6fd9d92/users/927f21ef1ca74ec58f85181728933e82/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:25.922356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5048d-7d1b-47d5-9cb3-2f9b6fba3f38 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:25.922642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5048d-7d1b-47d5-9cb3-2f9b6fba3f38 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:25.922915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5048d-7d1b-47d5-9cb3-2f9b6fba3f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:25.941024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-eed5048d-7d1b-47d5-9cb3-2f9b6fba3f38 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:25.941196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:25.942216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-eed5048d-7d1b-47d5-9cb3-2f9b6fba3f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:25.974870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2db46-a7b5-45d9-aa2a-f10131887223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:25.976586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2db46-a7b5-45d9-aa2a-f10131887223 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:25.987060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 818/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:25.987890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2db46-a7b5-45d9-aa2a-f10131887223 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:25.998601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2db46-a7b5-45d9-aa2a-f10131887223 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.012008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 801/1619] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:26.013141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5048d-7d1b-47d5-9cb3-2f9b6fba3f38 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.014779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-eed5048d-7d1b-47d5-9cb3-2f9b6fba3f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:26.067832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb27a-c0f7-4d02-ad60-4fc8b9691422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.068574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb27a-c0f7-4d02-ad60-4fc8b9691422 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.068870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb27a-c0f7-4d02-ad60-4fc8b9691422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.069148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb27a-c0f7-4d02-ad60-4fc8b9691422 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.069807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eb27a-c0f7-4d02-ad60-4fc8b9691422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.071886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eb27a-c0f7-4d02-ad60-4fc8b9691422 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.076903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5048d-7d1b-47d5-9cb3-2f9b6fba3f38 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.080679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ee9cf-f65c-487f-aca1-dee1bbede15a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.081356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ee9cf-f65c-487f-aca1-dee1bbede15a service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.081642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ee9cf-f65c-487f-aca1-dee1bbede15a service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.082182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ee9cf-f65c-487f-aca1-dee1bbede15a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.096851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ee9cf-f65c-487f-aca1-dee1bbede15a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.098873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ee9cf-f65c-487f-aca1-dee1bbede15a service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.105647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 819/1620] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:25 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:26.105647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb27a-c0f7-4d02-ad60-4fc8b9691422 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.117317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb27a-c0f7-4d02-ad60-4fc8b9691422 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.119658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 802/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:26.122521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ee9cf-f65c-487f-aca1-dee1bbede15a service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.123546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a70c28-3a05-4e15-8345-ca01559269c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.124219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70c28-3a05-4e15-8345-ca01559269c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.124503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70c28-3a05-4e15-8345-ca01559269c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.124814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70c28-3a05-4e15-8345-ca01559269c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.125750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a70c28-3a05-4e15-8345-ca01559269c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.127798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a70c28-3a05-4e15-8345-ca01559269c2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.170415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 820/1622] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:26 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:26.172522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a01bb3-c595-4a45-9c4d-d627d499d0ec None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.172969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a01bb3-c595-4a45-9c4d-d627d499d0ec None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.173321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a01bb3-c595-4a45-9c4d-d627d499d0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.196527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-95a01bb3-c595-4a45-9c4d-d627d499d0ec None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:26.197771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-95a01bb3-c595-4a45-9c4d-d627d499d0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:26.200533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ee9cf-f65c-487f-aca1-dee1bbede15a service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.237852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:26.256970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44483fd4-0559-40ad-8de1-183224814c78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.257630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483fd4-0559-40ad-8de1-183224814c78 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.257936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483fd4-0559-40ad-8de1-183224814c78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.258211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483fd4-0559-40ad-8de1-183224814c78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.275464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 821/1623] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:26.279317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a01bb3-c595-4a45-9c4d-d627d499d0ec None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.321432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:26.335074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44483fd4-0559-40ad-8de1-183224814c78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.336941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44483fd4-0559-40ad-8de1-183224814c78 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.358416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 803/1624] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:26.359437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44483fd4-0559-40ad-8de1-183224814c78 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.372519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44483fd4-0559-40ad-8de1-183224814c78 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.377329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a01bb3-c595-4a45-9c4d-d627d499d0ec None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.377753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f05195-5bac-4b2c-8fc1-22c2be9012b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.378413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05195-5bac-4b2c-8fc1-22c2be9012b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.378644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05195-5bac-4b2c-8fc1-22c2be9012b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.378917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05195-5bac-4b2c-8fc1-22c2be9012b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.379786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f05195-5bac-4b2c-8fc1-22c2be9012b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.381428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f05195-5bac-4b2c-8fc1-22c2be9012b8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.387871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607864cf-a590-4ede-8848-d0dd201fc970 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.389114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607864cf-a590-4ede-8848-d0dd201fc970 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.389732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607864cf-a590-4ede-8848-d0dd201fc970 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.390364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607864cf-a590-4ede-8848-d0dd201fc970 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.397385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 804/1625] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:26.398317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f05195-5bac-4b2c-8fc1-22c2be9012b8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.418243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f05195-5bac-4b2c-8fc1-22c2be9012b8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.432450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc56ee-d1f1-45d5-b564-258c1d0b56d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.434796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc56ee-d1f1-45d5-b564-258c1d0b56d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.436760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc56ee-d1f1-45d5-b564-258c1d0b56d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.437360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc56ee-d1f1-45d5-b564-258c1d0b56d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.438551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc56ee-d1f1-45d5-b564-258c1d0b56d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.441393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc56ee-d1f1-45d5-b564-258c1d0b56d9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.489899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 805/1626] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:26 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:26.491936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc004-91f2-4047-91d1-8a09988a11d1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.492629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc004-91f2-4047-91d1-8a09988a11d1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.493186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc004-91f2-4047-91d1-8a09988a11d1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.497793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 806/1627] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 17:25:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:25:26.512248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dc004-91f2-4047-91d1-8a09988a11d1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.545355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607864cf-a590-4ede-8848-d0dd201fc970 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.547830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607864cf-a590-4ede-8848-d0dd201fc970 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.571059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 822/1628] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:26.572042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607864cf-a590-4ede-8848-d0dd201fc970 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.599005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607864cf-a590-4ede-8848-d0dd201fc970 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.603299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dc004-91f2-4047-91d1-8a09988a11d1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.608661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dc33a-c9e1-4186-aecd-5ab397b0caa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.609547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dc33a-c9e1-4186-aecd-5ab397b0caa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.610369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dc33a-c9e1-4186-aecd-5ab397b0caa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.610844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dc33a-c9e1-4186-aecd-5ab397b0caa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.611802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2342278-6b3d-487b-9f54-6468c35b1119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.612704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342278-6b3d-487b-9f54-6468c35b1119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.612983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342278-6b3d-487b-9f54-6468c35b1119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.613257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342278-6b3d-487b-9f54-6468c35b1119 admin admin] PATH_INFO: `/v3/projects/5a81d512823043b199302ce58d8ce45c/users/e2e7660a1a5f43259c2b575c9ccaba59/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.629031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094dc33a-c9e1-4186-aecd-5ab397b0caa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.632878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094dc33a-c9e1-4186-aecd-5ab397b0caa5 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.647515 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2342278-6b3d-487b-9f54-6468c35b1119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a81d512823043b199302ce58d8ce45c, user_id=e2e7660a1a5f43259c2b575c9ccaba59, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.657034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2342278-6b3d-487b-9f54-6468c35b1119 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.657923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 807/1629] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:26.658790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dc33a-c9e1-4186-aecd-5ab397b0caa5 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.669390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dc33a-c9e1-4186-aecd-5ab397b0caa5 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.677785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1561e7-be73-4c91-9e6b-95c0aecafbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.678464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1561e7-be73-4c91-9e6b-95c0aecafbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.678747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1561e7-be73-4c91-9e6b-95c0aecafbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.679022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1561e7-be73-4c91-9e6b-95c0aecafbd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.679811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1561e7-be73-4c91-9e6b-95c0aecafbd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.681494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1561e7-be73-4c91-9e6b-95c0aecafbd1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.686286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 823/1630] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:26 2022] PUT /identity/v3/projects/5a81d512823043b199302ce58d8ce45c/users/e2e7660a1a5f43259c2b575c9ccaba59/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:26.688335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37975c-dc28-4878-8216-fa2d72b3a27d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.688628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37975c-dc28-4878-8216-fa2d72b3a27d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.688914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37975c-dc28-4878-8216-fa2d72b3a27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.710228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 808/1631] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:26.711435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1561e7-be73-4c91-9e6b-95c0aecafbd1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.720662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2a37975c-dc28-4878-8216-fa2d72b3a27d None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:26.721905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2a37975c-dc28-4878-8216-fa2d72b3a27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:26.743141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1561e7-be73-4c91-9e6b-95c0aecafbd1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.854310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 824/1632] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:25:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:26.855314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37975c-dc28-4878-8216-fa2d72b3a27d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.863196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:26.878462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dc8c1-e7ca-4be5-96a4-488d5dcaa3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.879839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc8c1-e7ca-4be5-96a4-488d5dcaa3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.880144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc8c1-e7ca-4be5-96a4-488d5dcaa3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.880508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc8c1-e7ca-4be5-96a4-488d5dcaa3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:26.881429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dc8c1-e7ca-4be5-96a4-488d5dcaa3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:26.883965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dc8c1-e7ca-4be5-96a4-488d5dcaa3a8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:26.902459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 809/1633] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:26.904234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dc8c1-e7ca-4be5-96a4-488d5dcaa3a8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:26.942218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:26.976823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37975c-dc28-4878-8216-fa2d72b3a27d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:26.982908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ecbb3-81c4-49b7-813b-1c331f77b5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:26.983856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ecbb3-81c4-49b7-813b-1c331f77b5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:26.984138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ecbb3-81c4-49b7-813b-1c331f77b5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:26.984410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ecbb3-81c4-49b7-813b-1c331f77b5ee admin admin] PATH_INFO: `/v3/projects/5933adf3d9fe4731b4dc5ef0f9254aa3/users/9505b8add02e424bb4ed2dd50b5a247f/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.004265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dc8c1-e7ca-4be5-96a4-488d5dcaa3a8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:27.008012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87da56-db17-45b5-bf12-9d26f0b78cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:27.008844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87da56-db17-45b5-bf12-9d26f0b78cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.009708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87da56-db17-45b5-bf12-9d26f0b78cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.009708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87da56-db17-45b5-bf12-9d26f0b78cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.010156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ecbb3-81c4-49b7-813b-1c331f77b5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5933adf3d9fe4731b4dc5ef0f9254aa3, user_id=9505b8add02e424bb4ed2dd50b5a247f, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:27.014095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ecbb3-81c4-49b7-813b-1c331f77b5ee admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:27.025047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87da56-db17-45b5-bf12-9d26f0b78cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:27.028468 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87da56-db17-45b5-bf12-9d26f0b78cf2 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:27.032401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 825/1634] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:26 2022] PUT /identity/v3/projects/5933adf3d9fe4731b4dc5ef0f9254aa3/users/9505b8add02e424bb4ed2dd50b5a247f/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:27.036445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ecbb3-81c4-49b7-813b-1c331f77b5ee admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:27.047471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 810/1635] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:27.049030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215211-9925-4757-9aeb-aa40dcdb22a5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.049352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215211-9925-4757-9aeb-aa40dcdb22a5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.049728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215211-9925-4757-9aeb-aa40dcdb22a5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.051625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 811/1636] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 17:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:25:27.052394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215211-9925-4757-9aeb-aa40dcdb22a5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:27.128825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ecbb3-81c4-49b7-813b-1c331f77b5ee admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:27.134884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992511d-e0c9-4698-8b00-1fafa13f9b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:27.138536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992511d-e0c9-4698-8b00-1fafa13f9b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.138819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992511d-e0c9-4698-8b00-1fafa13f9b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.139101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992511d-e0c9-4698-8b00-1fafa13f9b10 admin admin] PATH_INFO: `/v3/projects/5a81d512823043b199302ce58d8ce45c/users/e2e7660a1a5f43259c2b575c9ccaba59/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.142427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992511d-e0c9-4698-8b00-1fafa13f9b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a81d512823043b199302ce58d8ce45c, user_id=e2e7660a1a5f43259c2b575c9ccaba59, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:27.146475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992511d-e0c9-4698-8b00-1fafa13f9b10 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:27.160045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 826/1637] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:27 2022] PUT /identity/v3/projects/5a81d512823043b199302ce58d8ce45c/users/e2e7660a1a5f43259c2b575c9ccaba59/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:27.161418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531145ae-657f-4472-b652-9571094026a7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.161706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531145ae-657f-4472-b652-9571094026a7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.161979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531145ae-657f-4472-b652-9571094026a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.187774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-531145ae-657f-4472-b652-9571094026a7 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:27.188967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-531145ae-657f-4472-b652-9571094026a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:27.296578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215211-9925-4757-9aeb-aa40dcdb22a5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:27.453617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 827/1638] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:25:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:27.466549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:27.467690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.468206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.469113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.470102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:27.471979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:27.474738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5c19-9e66-4c56-b367-016321ee0749 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.475034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5c19-9e66-4c56-b367-016321ee0749 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.476830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5c19-9e66-4c56-b367-016321ee0749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.506819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 812/1639] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:27.508154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:27.513289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:27.555366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-79ab5c19-9e66-4c56-b367-016321ee0749 None None] MFA Rules not processed for user `e2e7660a1a5f43259c2b575c9ccaba59`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:27.556612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-79ab5c19-9e66-4c56-b367-016321ee0749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:27.610029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:27.644555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e0ec2-7e27-429f-a6d8-be9e5960433b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:27.654141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:27.656366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b6c17-f228-4990-8127-e7aa79d3d535 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:27.656366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b6c17-f228-4990-8127-e7aa79d3d535 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.656749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b6c17-f228-4990-8127-e7aa79d3d535 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.657134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b6c17-f228-4990-8127-e7aa79d3d535 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.685985 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b6c17-f228-4990-8127-e7aa79d3d535 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:27.687545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b6c17-f228-4990-8127-e7aa79d3d535 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:27.718291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 813/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:27.720891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b6c17-f228-4990-8127-e7aa79d3d535 service glance] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:27.748556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b6c17-f228-4990-8127-e7aa79d3d535 service glance] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:27.761081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb2aea-47d5-47f4-819c-70638b1b6ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:27.762774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb2aea-47d5-47f4-819c-70638b1b6ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.763620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb2aea-47d5-47f4-819c-70638b1b6ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.763967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb2aea-47d5-47f4-819c-70638b1b6ac0 admin admin] PATH_INFO: `/v3/projects/5933adf3d9fe4731b4dc5ef0f9254aa3/users/9505b8add02e424bb4ed2dd50b5a247f/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.772825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb2aea-47d5-47f4-819c-70638b1b6ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5933adf3d9fe4731b4dc5ef0f9254aa3, user_id=9505b8add02e424bb4ed2dd50b5a247f, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:27.775835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb2aea-47d5-47f4-819c-70638b1b6ac0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:27.799333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 814/1641] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:27 2022] PUT /identity/v3/projects/5933adf3d9fe4731b4dc5ef0f9254aa3/users/9505b8add02e424bb4ed2dd50b5a247f/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:27.811973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ce281-bf61-49e2-ba48-3e30323d807d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:27.812670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ce281-bf61-49e2-ba48-3e30323d807d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:27.812670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ce281-bf61-49e2-ba48-3e30323d807d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:27.825407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 828/1642] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:27.838387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab5c19-9e66-4c56-b367-016321ee0749 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:27.889304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5c5ce281-bf61-49e2-ba48-3e30323d807d None None] MFA Rules not processed for user `9505b8add02e424bb4ed2dd50b5a247f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:27.890547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5c5ce281-bf61-49e2-ba48-3e30323d807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:28.053654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab5c19-9e66-4c56-b367-016321ee0749 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:28.071813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 815/1643] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:28.084629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ce281-bf61-49e2-ba48-3e30323d807d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:28.198880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d1ddf-e119-4b22-a1b6-d3093880aceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:28.200451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d1ddf-e119-4b22-a1b6-d3093880aceb service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:28.200451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d1ddf-e119-4b22-a1b6-d3093880aceb service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:28.200451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d1ddf-e119-4b22-a1b6-d3093880aceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:28.265382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ce281-bf61-49e2-ba48-3e30323d807d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:28.278209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9d1d3-de43-4615-9a85-ee839eed34fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:28.278209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9d1d3-de43-4615-9a85-ee839eed34fd service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:28.278209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9d1d3-de43-4615-9a85-ee839eed34fd service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:28.278459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9d1d3-de43-4615-9a85-ee839eed34fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:28.293644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d1ddf-e119-4b22-a1b6-d3093880aceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:28.297991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d1ddf-e119-4b22-a1b6-d3093880aceb service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:28.311008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 829/1644] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:25:28.312100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d1ddf-e119-4b22-a1b6-d3093880aceb service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:28.332826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d1ddf-e119-4b22-a1b6-d3093880aceb service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:28.352047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8f0ad-8e8f-4ab1-bffc-5a41033feb12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:28.352913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f0ad-8e8f-4ab1-bffc-5a41033feb12 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:28.353201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f0ad-8e8f-4ab1-bffc-5a41033feb12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:28.359230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f0ad-8e8f-4ab1-bffc-5a41033feb12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:28.380011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8f0ad-8e8f-4ab1-bffc-5a41033feb12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:28.381784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8f0ad-8e8f-4ab1-bffc-5a41033feb12 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:28.412337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 830/1645] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:28 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:28.413730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e715e5-eacd-4a7d-813e-c535b43d1e49 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:28.414022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e715e5-eacd-4a7d-813e-c535b43d1e49 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:28.414307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e715e5-eacd-4a7d-813e-c535b43d1e49 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:28.422355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 831/1646] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 17:25:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:25:28.433267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5828f0e-07eb-4e7d-b76d-312127c09b2b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:28.433550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5828f0e-07eb-4e7d-b76d-312127c09b2b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:28.433844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5828f0e-07eb-4e7d-b76d-312127c09b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:28.462428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:28.499783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9d1d3-de43-4615-9a85-ee839eed34fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:28.499950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9d1d3-de43-4615-9a85-ee839eed34fd service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:28.504112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a5828f0e-07eb-4e7d-b76d-312127c09b2b None None] MFA Rules not processed for user `71a48b3b4fa94798b158488f08bae743`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:28.505938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a5828f0e-07eb-4e7d-b76d-312127c09b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:28.520711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:28.520859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 816/1647] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:25:28.606661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 832/1648] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:25:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:28.624823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9d1d3-de43-4615-9a85-ee839eed34fd service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:28.637534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9d1d3-de43-4615-9a85-ee839eed34fd service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:28.651160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9355c6-7ecb-4245-8a3e-7f4296a53696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:28.652394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9355c6-7ecb-4245-8a3e-7f4296a53696 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:28.652679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9355c6-7ecb-4245-8a3e-7f4296a53696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:28.652954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9355c6-7ecb-4245-8a3e-7f4296a53696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:28.683003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9355c6-7ecb-4245-8a3e-7f4296a53696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:28.684787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9355c6-7ecb-4245-8a3e-7f4296a53696 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:28.710102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 817/1649] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:28.756489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5828f0e-07eb-4e7d-b76d-312127c09b2b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:28.954976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5828f0e-07eb-4e7d-b76d-312127c09b2b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:28.989389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:28.995040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:28.997639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:28.998223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] PATH_INFO: `/v3/users/93efc8c9018e4b3496b127a410a3aecb` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:29.003721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] RBAC: Authorizing `identity:delete_user(user_id=93efc8c9018e4b3496b127a410a3aecb)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:29.015746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:29.092188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9355c6-7ecb-4245-8a3e-7f4296a53696 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:29.111143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9355c6-7ecb-4245-8a3e-7f4296a53696 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:29.112251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93efc8c9018e4b3496b127a410a3aecb'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.115043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93efc8c9018e4b3496b127a410a3aecb'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.131548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:29.134353 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:29.134878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:29.136655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] PATH_INFO: `/v3/users/3fd43777943042058151ec9ae8479155` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:29.137158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93efc8c9018e4b3496b127a410a3aecb'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.139184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd43777943042058151ec9ae8479155)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:29.140889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:25:29.140889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 28 17:25:29.140889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: May 28 17:25:29.143325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:29.171482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93efc8c9018e4b3496b127a410a3aecb'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.179731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e681b484-7d12-4eb7-a513-1710ac38d77f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93efc8c9018e4b3496b127a410a3aecb'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.218882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 833/1650] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:28 2022] DELETE /identity/v3/users/93efc8c9018e4b3496b127a410a3aecb => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:29.221562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7688-cab2-4e08-a5c9-493e4b6c0c59 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:29.223010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7688-cab2-4e08-a5c9-493e4b6c0c59 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:29.223010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7688-cab2-4e08-a5c9-493e4b6c0c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:29.289223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-24ac7688-cab2-4e08-a5c9-493e4b6c0c59 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:29.290717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-24ac7688-cab2-4e08-a5c9-493e4b6c0c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:29.298233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd43777943042058151ec9ae8479155'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.309885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd43777943042058151ec9ae8479155'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.336844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:29.341063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd43777943042058151ec9ae8479155'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.345063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd43777943042058151ec9ae8479155'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.361008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd43777943042058151ec9ae8479155'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.373537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 818/1651] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:29 2022] DELETE /identity/v3/users/3fd43777943042058151ec9ae8479155 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:29.373537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:29.375141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 834/1652] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:29.376872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:29.378741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c0d8-89b4-4872-a345-0d43d30f9146 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:29.379197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c0d8-89b4-4872-a345-0d43d30f9146 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:29.379519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c0d8-89b4-4872-a345-0d43d30f9146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:29.407448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6771c0d8-89b4-4872-a345-0d43d30f9146 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:29.409499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6771c0d8-89b4-4872-a345-0d43d30f9146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:29.494848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2f81-c29e-4d51-8735-7d838482d2b5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:29.501755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:29.503045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:29.503471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:29.503958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] PATH_INFO: `/v3/users/8c0632580b96498c9b010a80e9e0f659` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:29.505284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0632580b96498c9b010a80e9e0f659)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:29.508134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:29.560857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 835/1653] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:29.562131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771c0d8-89b4-4872-a345-0d43d30f9146 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:29.564744 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6771c0d8-89b4-4872-a345-0d43d30f9146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:29.580835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0632580b96498c9b010a80e9e0f659'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.584488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:29.597094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0632580b96498c9b010a80e9e0f659'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.656871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0632580b96498c9b010a80e9e0f659'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.660170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0632580b96498c9b010a80e9e0f659'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.736681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0632580b96498c9b010a80e9e0f659'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:29.763325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 819/1654] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:29 2022] DELETE /identity/v3/users/8c0632580b96498c9b010a80e9e0f659 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:29.764311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:29.772258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771c0d8-89b4-4872-a345-0d43d30f9146 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:29.781280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:29.895613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a716e28-d62c-4ab0-acb2-ade1418dda0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:29.896478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716e28-d62c-4ab0-acb2-ade1418dda0f service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:29.896478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716e28-d62c-4ab0-acb2-ade1418dda0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:29.896663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716e28-d62c-4ab0-acb2-ade1418dda0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:29.902087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1a716e28-d62c-4ab0-acb2-ade1418dda0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:29.950594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8779510-c6e9-4907-b51e-10d4ffea53cc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:29.975351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234a3bd-4d4a-48c0-a00c-435531236356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:29.976349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234a3bd-4d4a-48c0-a00c-435531236356 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:29.979770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234a3bd-4d4a-48c0-a00c-435531236356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:29.980236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234a3bd-4d4a-48c0-a00c-435531236356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:29.990694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4234a3bd-4d4a-48c0-a00c-435531236356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:30.032453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:30.072051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a716e28-d62c-4ab0-acb2-ade1418dda0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:30.074093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a716e28-d62c-4ab0-acb2-ade1418dda0f service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:30.097768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 836/1655] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:30.101335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57db08-4002-40cc-a3b7-d3b22b68d3c3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:30.101335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57db08-4002-40cc-a3b7-d3b22b68d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:30.102688 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57db08-4002-40cc-a3b7-d3b22b68d3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:30.224746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9f57db08-4002-40cc-a3b7-d3b22b68d3c3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:30.226795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9f57db08-4002-40cc-a3b7-d3b22b68d3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:30.237561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 837/1656] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:30.274675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57db08-4002-40cc-a3b7-d3b22b68d3c3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:30.290791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234a3bd-4d4a-48c0-a00c-435531236356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:30.298459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234a3bd-4d4a-48c0-a00c-435531236356 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:30.311905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57db08-4002-40cc-a3b7-d3b22b68d3c3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:30.329123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d165463-bab3-45ed-b4d6-61cf0b82e10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:30.331006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d165463-bab3-45ed-b4d6-61cf0b82e10a service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:30.331780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 820/1657] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:30.336157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d165463-bab3-45ed-b4d6-61cf0b82e10a service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:30.336622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d165463-bab3-45ed-b4d6-61cf0b82e10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:30.362350 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5c202-7363-4579-8817-9695a6c9730b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:30.362747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5c202-7363-4579-8817-9695a6c9730b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:30.363123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5c202-7363-4579-8817-9695a6c9730b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:30.365076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 821/1658] 10.222.0.33 () {54 vars in 960 bytes} [Sat May 28 17:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 17:25:30.375180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79885c94-6509-47b7-9242-1f61b730c8b2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:30.376236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79885c94-6509-47b7-9242-1f61b730c8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:30.376652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79885c94-6509-47b7-9242-1f61b730c8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:30.376789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d165463-bab3-45ed-b4d6-61cf0b82e10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:30.385343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d165463-bab3-45ed-b4d6-61cf0b82e10a service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:30.425932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-79885c94-6509-47b7-9242-1f61b730c8b2 None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:30.428794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-79885c94-6509-47b7-9242-1f61b730c8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:30.430142 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 838/1659] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:30.653869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d165463-bab3-45ed-b4d6-61cf0b82e10a service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:30.655748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 822/1660] 10.222.0.33 () {58 vars in 1057 bytes} [Sat May 28 17:25:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:30.669870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d165463-bab3-45ed-b4d6-61cf0b82e10a service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:30.691812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:30.692417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:30.692705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:30.692986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:30.697757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:30.744678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:30.748338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:30.779581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 839/1661] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:25:31.449481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79885c94-6509-47b7-9242-1f61b730c8b2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:31.450925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-79885c94-6509-47b7-9242-1f61b730c8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:31.477306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79885c94-6509-47b7-9242-1f61b730c8b2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:31.489230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:31.490128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:31.490410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:31.490684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] PATH_INFO: `/v3/projects/eeb3e12d39cb49e3bbac2c7d030f8995` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:31.492100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeb3e12d39cb49e3bbac2c7d030f8995)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:31.494307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:31.512830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:31.618360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeb3e12d39cb49e3bbac2c7d030f8995', 'request_id': 'req-5755754c-8be3-4bd7-8c48-56f61318855a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:31.640209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeb3e12d39cb49e3bbac2c7d030f8995', 'request_id': 'req-5755754c-8be3-4bd7-8c48-56f61318855a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:31.654582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeb3e12d39cb49e3bbac2c7d030f8995', 'request_id': 'req-5755754c-8be3-4bd7-8c48-56f61318855a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:31.710616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] The token cache is being invalidate because project eeb3e12d39cb49e3bbac2c7d030f8995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:31.710923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:31.716581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:31.719626 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 823/1662] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:31 2022] DELETE /identity/v3/projects/eeb3e12d39cb49e3bbac2c7d030f8995 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:31.726942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:31.745324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3494d86-4022-429a-9467-e91b261203f2 service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:31.884729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:31.886335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:31.886631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:31.886908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] PATH_INFO: `/v3/users/9505b8add02e424bb4ed2dd50b5a247f` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:31.888880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9505b8add02e424bb4ed2dd50b5a247f)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:31.892778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:32.077289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9505b8add02e424bb4ed2dd50b5a247f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.079146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9505b8add02e424bb4ed2dd50b5a247f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.117435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9505b8add02e424bb4ed2dd50b5a247f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.137916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9505b8add02e424bb4ed2dd50b5a247f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.139012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9505b8add02e424bb4ed2dd50b5a247f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.172511 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 840/1663] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:31 2022] DELETE /identity/v3/users/9505b8add02e424bb4ed2dd50b5a247f => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:32.265830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:32.267525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:32.290352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755754c-8be3-4bd7-8c48-56f61318855a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:32.432822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:32.434317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:32.435000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:32.435521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] PATH_INFO: `/v3/projects/29920bb776f840388b047e19dbac990f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:32.437241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29920bb776f840388b047e19dbac990f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:32.439849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:32.514435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29920bb776f840388b047e19dbac990f', 'request_id': 'req-7e37a6c8-61cf-44d3-b437-a0f118282da7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.532339 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29920bb776f840388b047e19dbac990f', 'request_id': 'req-7e37a6c8-61cf-44d3-b437-a0f118282da7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.542746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29920bb776f840388b047e19dbac990f', 'request_id': 'req-7e37a6c8-61cf-44d3-b437-a0f118282da7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.548834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:32.550579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:32.593356 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd674cfb-d37d-410f-b27c-d7a68c5d75aa admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:32.594414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] The token cache is being invalidate because project 29920bb776f840388b047e19dbac990f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:32.594714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.595988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.603048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 824/1664] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:32 2022] DELETE /identity/v3/projects/29920bb776f840388b047e19dbac990f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:32.606238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:32.608155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:32.685334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:32.720470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:32.739594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37a6c8-61cf-44d3-b437-a0f118282da7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:32.748082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:32.749234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:32.749678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:32.750100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] PATH_INFO: `/v3/users/e2e7660a1a5f43259c2b575c9ccaba59` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:32.751575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e7660a1a5f43259c2b575c9ccaba59)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:32.753794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:32.780089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:32.782221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:32.782724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:32.783281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] PATH_INFO: `/v3/projects/24240be31473431cb8690f8cb08a3cd5` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:32.785694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24240be31473431cb8690f8cb08a3cd5)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:32.788230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 17:25:32.788230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: warnings.warn(msg) May 28 17:25:32.788230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: May 28 17:25:32.790132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:32.863077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e7660a1a5f43259c2b575c9ccaba59'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.867056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24240be31473431cb8690f8cb08a3cd5', 'request_id': 'req-41b89604-e832-4c3b-93d0-42576929bdd8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.896004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e7660a1a5f43259c2b575c9ccaba59'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.897774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24240be31473431cb8690f8cb08a3cd5', 'request_id': 'req-41b89604-e832-4c3b-93d0-42576929bdd8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.911417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24240be31473431cb8690f8cb08a3cd5', 'request_id': 'req-41b89604-e832-4c3b-93d0-42576929bdd8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.918225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e7660a1a5f43259c2b575c9ccaba59'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.919895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e7660a1a5f43259c2b575c9ccaba59'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.940415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6c1ed289-b703-48ac-abbe-ac4b6a31564f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e7660a1a5f43259c2b575c9ccaba59'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.940856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] The token cache is being invalidate because project 24240be31473431cb8690f8cb08a3cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:32.941827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.943478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-41b89604-e832-4c3b-93d0-42576929bdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:32.951256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 841/1665] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:32 2022] DELETE /identity/v3/projects/24240be31473431cb8690f8cb08a3cd5 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:25:32.956035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5611ff-248b-4942-bc61-c6642f027d88 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:32.956333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5611ff-248b-4942-bc61-c6642f027d88 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:32.956619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5611ff-248b-4942-bc61-c6642f027d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:32.960495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 825/1666] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:32 2022] DELETE /identity/v3/users/e2e7660a1a5f43259c2b575c9ccaba59 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:32.984927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5e5611ff-248b-4942-bc61-c6642f027d88 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:32.986329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5e5611ff-248b-4942-bc61-c6642f027d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:33.075899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 842/1667] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:33.078667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90a1a2-5f01-4845-9b7d-20c0b40de828 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.079047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90a1a2-5f01-4845-9b7d-20c0b40de828 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.079435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90a1a2-5f01-4845-9b7d-20c0b40de828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.103149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fe90a1a2-5f01-4845-9b7d-20c0b40de828 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:33.105217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fe90a1a2-5f01-4845-9b7d-20c0b40de828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:33.128056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 826/1668] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:33.131690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5611ff-248b-4942-bc61-c6642f027d88 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:33.152503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5611ff-248b-4942-bc61-c6642f027d88 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:33.163270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:33.164489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.164782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.165283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.166557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:33.167683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:33.169795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:33.192628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 843/1669] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:33.195664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90a1a2-5f01-4845-9b7d-20c0b40de828 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:33.205751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90a1a2-5f01-4845-9b7d-20c0b40de828 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:33.211792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823f150-e3a0-4bbf-9b8e-eb933fdc0d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:33.212468 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823f150-e3a0-4bbf-9b8e-eb933fdc0d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.212749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823f150-e3a0-4bbf-9b8e-eb933fdc0d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.214008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823f150-e3a0-4bbf-9b8e-eb933fdc0d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.215022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823f150-e3a0-4bbf-9b8e-eb933fdc0d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:33.217102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823f150-e3a0-4bbf-9b8e-eb933fdc0d38 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:33.283184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:33.290916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 827/1670] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:33 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:33.298028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409a93b-1c62-4332-afdb-79de20ba7930 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:33.394212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:33.419552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534839d3-6295-42c3-9a7e-29e494d7dcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:33.420530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534839d3-6295-42c3-9a7e-29e494d7dcc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.420910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534839d3-6295-42c3-9a7e-29e494d7dcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.421276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534839d3-6295-42c3-9a7e-29e494d7dcc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.422157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534839d3-6295-42c3-9a7e-29e494d7dcc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:33.424288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534839d3-6295-42c3-9a7e-29e494d7dcc7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:33.463482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 844/1671] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:33 2022] POST /identity/v3/users => generated 282 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:33.468881 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823f150-e3a0-4bbf-9b8e-eb933fdc0d38 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:33.484673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823f150-e3a0-4bbf-9b8e-eb933fdc0d38 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:33.501257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec9b6d-d1d8-440c-9f89-c8bb10929517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:33.502199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec9b6d-d1d8-440c-9f89-c8bb10929517 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.502849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec9b6d-d1d8-440c-9f89-c8bb10929517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.503286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec9b6d-d1d8-440c-9f89-c8bb10929517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.504469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec9b6d-d1d8-440c-9f89-c8bb10929517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:33.508599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec9b6d-d1d8-440c-9f89-c8bb10929517 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:33.535590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534839d3-6295-42c3-9a7e-29e494d7dcc7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:33.536151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 828/1672] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:33.552803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534839d3-6295-42c3-9a7e-29e494d7dcc7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:33.568588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd54d6d-b31e-450a-93e3-4a7df4d1c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:33.570426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd54d6d-b31e-450a-93e3-4a7df4d1c06d admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.570914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd54d6d-b31e-450a-93e3-4a7df4d1c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.572706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd54d6d-b31e-450a-93e3-4a7df4d1c06d admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.577119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd54d6d-b31e-450a-93e3-4a7df4d1c06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:33.586248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd54d6d-b31e-450a-93e3-4a7df4d1c06d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:33.624129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 845/1673] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:33.628391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec9b6d-d1d8-440c-9f89-c8bb10929517 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:33.645376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec9b6d-d1d8-440c-9f89-c8bb10929517 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:33.658864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:33.661662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.662162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.663567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] PATH_INFO: `/v3/projects/c5549ad0ab514013be5b2334468bef72/users/44c72580324a4d90acc9e96209fb7663/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.698354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5549ad0ab514013be5b2334468bef72, user_id=44c72580324a4d90acc9e96209fb7663, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:33.702109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:33.731461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 829/1674] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:33 2022] PUT /identity/v3/projects/c5549ad0ab514013be5b2334468bef72/users/44c72580324a4d90acc9e96209fb7663/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:33.742543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd78a-654a-469c-b521-3c35a574a3f8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.742827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd78a-654a-469c-b521-3c35a574a3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.743115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd78a-654a-469c-b521-3c35a574a3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.767833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:33.769828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:33.819665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d59cd78a-654a-469c-b521-3c35a574a3f8 None None] MFA Rules not processed for user `44c72580324a4d90acc9e96209fb7663`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:33.821928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d59cd78a-654a-469c-b521-3c35a574a3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:33.870813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:33.939810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd165a-97c1-49f9-b804-c04a0d1a375a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:33.965813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:33.967271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:33.970928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:33.971684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] PATH_INFO: `/v3/projects/5a81d512823043b199302ce58d8ce45c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:33.978584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a81d512823043b199302ce58d8ce45c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:33.994518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:34.051648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 846/1675] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:33 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:34.052879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cd78a-654a-469c-b521-3c35a574a3f8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:34.060381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d59cd78a-654a-469c-b521-3c35a574a3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:34.092153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cd78a-654a-469c-b521-3c35a574a3f8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:34.103157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:34.104955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.106114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.106501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] PATH_INFO: `/v3/projects/5933adf3d9fe4731b4dc5ef0f9254aa3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.108461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5933adf3d9fe4731b4dc5ef0f9254aa3)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:34.112911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:34.126692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a81d512823043b199302ce58d8ce45c', 'request_id': 'req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.143730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a81d512823043b199302ce58d8ce45c', 'request_id': 'req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.152538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a81d512823043b199302ce58d8ce45c', 'request_id': 'req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.187917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] The token cache is being invalidate because project 5a81d512823043b199302ce58d8ce45c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:34.188348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.204503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.218399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 830/1676] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:33 2022] DELETE /identity/v3/projects/5a81d512823043b199302ce58d8ce45c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:25:34.218399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:34.218865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:34.262380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5933adf3d9fe4731b4dc5ef0f9254aa3', 'request_id': 'req-4b19be20-1b62-438f-8ef2-3a0ae38a9153'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.274734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5933adf3d9fe4731b4dc5ef0f9254aa3', 'request_id': 'req-4b19be20-1b62-438f-8ef2-3a0ae38a9153'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.294058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5933adf3d9fe4731b4dc5ef0f9254aa3', 'request_id': 'req-4b19be20-1b62-438f-8ef2-3a0ae38a9153'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.320201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] The token cache is being invalidate because project 5933adf3d9fe4731b4dc5ef0f9254aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:34.320642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.322092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4b19be20-1b62-438f-8ef2-3a0ae38a9153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:34.325607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 847/1677] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:34 2022] DELETE /identity/v3/projects/5933adf3d9fe4731b4dc5ef0f9254aa3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:34.329889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34384b-db4a-43fd-a238-ec691a067a6c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.330956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34384b-db4a-43fd-a238-ec691a067a6c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.332117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34384b-db4a-43fd-a238-ec691a067a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.376143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9a34384b-db4a-43fd-a238-ec691a067a6c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:34.378078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9a34384b-db4a-43fd-a238-ec691a067a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:34.400826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a598d-164d-4ed6-b2b2-5f0b4b9a7f40 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:34.469511 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:34.505808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 848/1678] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:34.507310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee62f7a-df56-4c8a-b6cd-2e7cfa4c1702 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.507737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee62f7a-df56-4c8a-b6cd-2e7cfa4c1702 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.508128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee62f7a-df56-4c8a-b6cd-2e7cfa4c1702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.537045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9ee62f7a-df56-4c8a-b6cd-2e7cfa4c1702 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:34.539331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9ee62f7a-df56-4c8a-b6cd-2e7cfa4c1702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:34.551678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 849/1679] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:34.553554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02789b4b-2097-494e-8a85-c8d3eab53489 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.554125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c1245-188b-456b-ac9f-f3cefdc77b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:34.554837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c1245-188b-456b-ac9f-f3cefdc77b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.554943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02789b4b-2097-494e-8a85-c8d3eab53489 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.555131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c1245-188b-456b-ac9f-f3cefdc77b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.555599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02789b4b-2097-494e-8a85-c8d3eab53489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.558477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c1245-188b-456b-ac9f-f3cefdc77b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.558691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c1245-188b-456b-ac9f-f3cefdc77b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:34.560797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c1245-188b-456b-ac9f-f3cefdc77b7e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:34.592453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-02789b4b-2097-494e-8a85-c8d3eab53489 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:34.593908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-02789b4b-2097-494e-8a85-c8d3eab53489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:34.606540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 850/1680] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:34.613799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1646f-fca1-4b46-8ac1-468a44d6daeb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.614273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1646f-fca1-4b46-8ac1-468a44d6daeb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.615543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1646f-fca1-4b46-8ac1-468a44d6daeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.627137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 831/1681] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:34 2022] POST /identity/v3/projects => generated 365 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:34.629814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c1245-188b-456b-ac9f-f3cefdc77b7e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:34.647604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:34.659576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a1a1646f-fca1-4b46-8ac1-468a44d6daeb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:34.662275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a1a1646f-fca1-4b46-8ac1-468a44d6daeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:34.805808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c1245-188b-456b-ac9f-f3cefdc77b7e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:34.814552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:34.825571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 851/1682] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:34.825571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1646f-fca1-4b46-8ac1-468a44d6daeb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:34.826706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a1a1646f-fca1-4b46-8ac1-468a44d6daeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:34.829903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.830204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.830481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.831252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:34.831846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:34.833800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:34.851329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1646f-fca1-4b46-8ac1-468a44d6daeb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:34.856945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 832/1683] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:34.858100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:34.864343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66288832-0604-46c7-a535-972185fbcd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:34.865237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66288832-0604-46c7-a535-972185fbcd9f admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.865567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66288832-0604-46c7-a535-972185fbcd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.866024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66288832-0604-46c7-a535-972185fbcd9f admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.867676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66288832-0604-46c7-a535-972185fbcd9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:34.872646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bc0e7-027d-4dc7-a3db-b68a11589a34 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:34.873324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66288832-0604-46c7-a535-972185fbcd9f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:34.879837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:34.880701 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.880997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.881277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.882017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:34.882330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:34.884025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:34.921562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 833/1684] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:34.921562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:34.921844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 852/1685] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:34 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:34.922830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66288832-0604-46c7-a535-972185fbcd9f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:34.940362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66288832-0604-46c7-a535-972185fbcd9f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:34.942068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab0b2-8d9d-4363-a816-d8504b13df7c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:34.950554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a6007-4602-42a3-a7e9-e054ae21c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:34.952516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bb2ca-d80d-4b39-a729-45db12191f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:34.953495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a6007-4602-42a3-a7e9-e054ae21c385 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.953952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a6007-4602-42a3-a7e9-e054ae21c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.954238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a6007-4602-42a3-a7e9-e054ae21c385 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.954952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a6007-4602-42a3-a7e9-e054ae21c385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:34.956918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a6007-4602-42a3-a7e9-e054ae21c385 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:34.958256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bb2ca-d80d-4b39-a729-45db12191f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:34.958723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bb2ca-d80d-4b39-a729-45db12191f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:34.959009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bb2ca-d80d-4b39-a729-45db12191f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:34.960719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bb2ca-d80d-4b39-a729-45db12191f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:34.962737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bb2ca-d80d-4b39-a729-45db12191f66 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:34.997554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 853/1686] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:34.998726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bb2ca-d80d-4b39-a729-45db12191f66 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.027588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bb2ca-d80d-4b39-a729-45db12191f66 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.036686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 834/1687] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:34 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:35.045515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a6007-4602-42a3-a7e9-e054ae21c385 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.192536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a6007-4602-42a3-a7e9-e054ae21c385 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.198778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906762cf-0d1a-46a7-8299-a74d409f2db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.200143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906762cf-0d1a-46a7-8299-a74d409f2db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.200534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906762cf-0d1a-46a7-8299-a74d409f2db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.200913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906762cf-0d1a-46a7-8299-a74d409f2db5 admin admin] PATH_INFO: `/v3/projects/edb20ec810d44704b0d0cf9e97cd0a9d/users/56620d1678404e4cb0f0f2c0abf33749/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.219670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974bffc-e2ec-46ae-b375-b34e653c8f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.222000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974bffc-e2ec-46ae-b375-b34e653c8f9a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.222291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974bffc-e2ec-46ae-b375-b34e653c8f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.225959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974bffc-e2ec-46ae-b375-b34e653c8f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.226266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906762cf-0d1a-46a7-8299-a74d409f2db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb20ec810d44704b0d0cf9e97cd0a9d, user_id=56620d1678404e4cb0f0f2c0abf33749, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.226658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974bffc-e2ec-46ae-b375-b34e653c8f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.229317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906762cf-0d1a-46a7-8299-a74d409f2db5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:35.233285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974bffc-e2ec-46ae-b375-b34e653c8f9a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:35.258355 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 835/1688] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:35 2022] PUT /identity/v3/projects/edb20ec810d44704b0d0cf9e97cd0a9d/users/56620d1678404e4cb0f0f2c0abf33749/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:35.261651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906762cf-0d1a-46a7-8299-a74d409f2db5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.306781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 854/1689] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:34 2022] POST /identity/v3/projects => generated 403 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:35.308742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974bffc-e2ec-46ae-b375-b34e653c8f9a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.406797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:35.410064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:35.452978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906762cf-0d1a-46a7-8299-a74d409f2db5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.494314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974bffc-e2ec-46ae-b375-b34e653c8f9a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.501307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e86a7-97e8-47c7-a460-7b941951ce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.501966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e86a7-97e8-47c7-a460-7b941951ce7c admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.511011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e86a7-97e8-47c7-a460-7b941951ce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.511740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e86a7-97e8-47c7-a460-7b941951ce7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.512569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e86a7-97e8-47c7-a460-7b941951ce7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.514256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e86a7-97e8-47c7-a460-7b941951ce7c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:35.532447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 855/1690] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:35.532447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e86a7-97e8-47c7-a460-7b941951ce7c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.553658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e86a7-97e8-47c7-a460-7b941951ce7c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.569587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309809d-02b4-4b48-b3bf-72b119575d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.570822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309809d-02b4-4b48-b3bf-72b119575d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.571109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309809d-02b4-4b48-b3bf-72b119575d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.571376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309809d-02b4-4b48-b3bf-72b119575d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.572385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309809d-02b4-4b48-b3bf-72b119575d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.577524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309809d-02b4-4b48-b3bf-72b119575d8b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:35.593232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad6794-d72f-472a-8673-f5bf9c4c5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.595545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad6794-d72f-472a-8673-f5bf9c4c5c13 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.596145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad6794-d72f-472a-8673-f5bf9c4c5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.596793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad6794-d72f-472a-8673-f5bf9c4c5c13 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.597665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad6794-d72f-472a-8673-f5bf9c4c5c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.600568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad6794-d72f-472a-8673-f5bf9c4c5c13 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:35.617148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 856/1691] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:35 2022] POST /identity/v3/users => generated 302 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:35.618039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309809d-02b4-4b48-b3bf-72b119575d8b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.640969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309809d-02b4-4b48-b3bf-72b119575d8b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.655333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 836/1692] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:35 2022] POST /identity/v3/users => generated 279 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:35.659228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad6794-d72f-472a-8673-f5bf9c4c5c13 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.661118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ec2ff-0366-4fe4-8265-08445726eebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.662032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec2ff-0366-4fe4-8265-08445726eebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.662310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec2ff-0366-4fe4-8265-08445726eebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.662583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec2ff-0366-4fe4-8265-08445726eebe admin admin] PATH_INFO: `/v3/projects/edb20ec810d44704b0d0cf9e97cd0a9d/users/56620d1678404e4cb0f0f2c0abf33749/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.670117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ec2ff-0366-4fe4-8265-08445726eebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb20ec810d44704b0d0cf9e97cd0a9d, user_id=56620d1678404e4cb0f0f2c0abf33749, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.672845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ec2ff-0366-4fe4-8265-08445726eebe admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:35.675641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad6794-d72f-472a-8673-f5bf9c4c5c13 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.681818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8bb97-9ea6-465a-bb8b-13cbdc772855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.683045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8bb97-9ea6-465a-bb8b-13cbdc772855 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.683621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8bb97-9ea6-465a-bb8b-13cbdc772855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.684071 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8bb97-9ea6-465a-bb8b-13cbdc772855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.685130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8bb97-9ea6-465a-bb8b-13cbdc772855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.687523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8bb97-9ea6-465a-bb8b-13cbdc772855 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:35.725299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 837/1693] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:35.728181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8bb97-9ea6-465a-bb8b-13cbdc772855 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.764217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8bb97-9ea6-465a-bb8b-13cbdc772855 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.884068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa00a9-5901-423a-b6d3-5622839b9e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.885927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa00a9-5901-423a-b6d3-5622839b9e15 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.886675 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa00a9-5901-423a-b6d3-5622839b9e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.887621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa00a9-5901-423a-b6d3-5622839b9e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.888750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa00a9-5901-423a-b6d3-5622839b9e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.890718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa00a9-5901-423a-b6d3-5622839b9e15 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:35.936577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 838/1694] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:35.937775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa00a9-5901-423a-b6d3-5622839b9e15 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:35.965068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa00a9-5901-423a-b6d3-5622839b9e15 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:35.981335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069ae6b-e14e-4098-8355-64304efe41fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:35.982264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ae6b-e14e-4098-8355-64304efe41fa admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:35.982658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ae6b-e14e-4098-8355-64304efe41fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:35.983040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ae6b-e14e-4098-8355-64304efe41fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:35.984009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069ae6b-e14e-4098-8355-64304efe41fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:35.985937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069ae6b-e14e-4098-8355-64304efe41fa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:36.007760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 839/1695] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:36.017857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069ae6b-e14e-4098-8355-64304efe41fa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:36.040278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 857/1696] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:35 2022] PUT /identity/v3/projects/edb20ec810d44704b0d0cf9e97cd0a9d/users/56620d1678404e4cb0f0f2c0abf33749/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:36.040278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ec2ff-0366-4fe4-8265-08445726eebe admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:36.040903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069ae6b-e14e-4098-8355-64304efe41fa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:36.048513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10400974-a15e-47d4-8007-361808ee38ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:36.057013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10400974-a15e-47d4-8007-361808ee38ab admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.058147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10400974-a15e-47d4-8007-361808ee38ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.058575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10400974-a15e-47d4-8007-361808ee38ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.059611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10400974-a15e-47d4-8007-361808ee38ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:36.061718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10400974-a15e-47d4-8007-361808ee38ab admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:36.069237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ec2ff-0366-4fe4-8265-08445726eebe admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:36.084240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a53087-f3ef-42c3-abdb-862935160488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:36.085156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a53087-f3ef-42c3-abdb-862935160488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.085431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a53087-f3ef-42c3-abdb-862935160488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.085707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a53087-f3ef-42c3-abdb-862935160488 admin admin] PATH_INFO: `/v3/projects/23a35593a4f44634b8f07986236611cf/users/5df621160a0a47ed8a1eb2c9e5645e63/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.093664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 840/1697] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:36.098611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44289a-69fe-429c-b0f5-0292b0c2de8a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.099645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44289a-69fe-429c-b0f5-0292b0c2de8a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.102918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44289a-69fe-429c-b0f5-0292b0c2de8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.124136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a53087-f3ef-42c3-abdb-862935160488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a35593a4f44634b8f07986236611cf, user_id=5df621160a0a47ed8a1eb2c9e5645e63, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:36.126824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a53087-f3ef-42c3-abdb-862935160488 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:36.170073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 858/1698] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:36 2022] PUT /identity/v3/projects/23a35593a4f44634b8f07986236611cf/users/5df621160a0a47ed8a1eb2c9e5645e63/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:36.171162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a53087-f3ef-42c3-abdb-862935160488 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:36.197606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-da44289a-69fe-429c-b0f5-0292b0c2de8a None None] MFA Rules not processed for user `56620d1678404e4cb0f0f2c0abf33749`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:36.200386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-da44289a-69fe-429c-b0f5-0292b0c2de8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:36.257447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:36.304280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a53087-f3ef-42c3-abdb-862935160488 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:36.326077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 841/1699] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:36.327743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c9fb-a001-415a-af23-88a8badff74a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.328265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c9fb-a001-415a-af23-88a8badff74a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.328744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c9fb-a001-415a-af23-88a8badff74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.333870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:36.334779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.335417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.335417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] PATH_INFO: `/v3/projects/af36a88886614de387192b0fa4b1870b/users/108fa4a2eb7f469fa152c5d0ca80e927/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.371689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af36a88886614de387192b0fa4b1870b, user_id=108fa4a2eb7f469fa152c5d0ca80e927, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:36.372876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9b04c9fb-a001-415a-af23-88a8badff74a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:36.374801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9b04c9fb-a001-415a-af23-88a8badff74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:36.378813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:36.383808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 842/1700] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:36.385056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd65e1-1d6c-4351-ac5f-07c41c75b029 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.385384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd65e1-1d6c-4351-ac5f-07c41c75b029 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.386409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd65e1-1d6c-4351-ac5f-07c41c75b029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.430227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-adbd65e1-1d6c-4351-ac5f-07c41c75b029 None None] MFA Rules not processed for user `7fcc62869ac34381a2327be46511137b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:36.431585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-adbd65e1-1d6c-4351-ac5f-07c41c75b029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:36.434795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 859/1701] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:36 2022] PUT /identity/v3/projects/af36a88886614de387192b0fa4b1870b/users/108fa4a2eb7f469fa152c5d0ca80e927/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:36.435836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:36.442050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:36.533253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a2bed-ca1f-4cd7-9cef-8f2d3cc34604 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:36.537543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cc694-d56f-45d4-bb2e-6e25abb402df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:36.538267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cc694-d56f-45d4-bb2e-6e25abb402df service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.538548 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cc694-d56f-45d4-bb2e-6e25abb402df service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.538867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cc694-d56f-45d4-bb2e-6e25abb402df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.544957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:36.611491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 843/1702] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:36.612952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd65e1-1d6c-4351-ac5f-07c41c75b029 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:36.614406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-adbd65e1-1d6c-4351-ac5f-07c41c75b029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:36.644353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cc694-d56f-45d4-bb2e-6e25abb402df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:36.645922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cc694-d56f-45d4-bb2e-6e25abb402df service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:36.658291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 860/1703] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:36.659756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966c6c2-5e1f-4f76-9712-2de46abe77c5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.660121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966c6c2-5e1f-4f76-9712-2de46abe77c5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.660475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966c6c2-5e1f-4f76-9712-2de46abe77c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.725992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e966c6c2-5e1f-4f76-9712-2de46abe77c5 None None] MFA Rules not processed for user `108fa4a2eb7f469fa152c5d0ca80e927`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:36.732715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e966c6c2-5e1f-4f76-9712-2de46abe77c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:36.757700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd65e1-1d6c-4351-ac5f-07c41c75b029 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:36.770181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c5954-bd53-4c04-af2d-9db85fb15e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:36.770826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c5954-bd53-4c04-af2d-9db85fb15e30 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:36.771115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c5954-bd53-4c04-af2d-9db85fb15e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:36.771412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c5954-bd53-4c04-af2d-9db85fb15e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:36.874983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 861/1704] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:36 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:36.876059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966c6c2-5e1f-4f76-9712-2de46abe77c5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:36.889866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e966c6c2-5e1f-4f76-9712-2de46abe77c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:36.927771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c5954-bd53-4c04-af2d-9db85fb15e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:36.929232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c5954-bd53-4c04-af2d-9db85fb15e30 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:36.960250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 844/1705] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:36.960932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c5954-bd53-4c04-af2d-9db85fb15e30 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:36.970548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:36.980553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c5954-bd53-4c04-af2d-9db85fb15e30 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:37.002150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bdabb-9c53-4226-bf10-42c14821a439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:37.002150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdabb-9c53-4226-bf10-42c14821a439 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:37.002150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdabb-9c53-4226-bf10-42c14821a439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:37.002150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdabb-9c53-4226-bf10-42c14821a439 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:37.003190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bdabb-9c53-4226-bf10-42c14821a439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:37.019646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bdabb-9c53-4226-bf10-42c14821a439 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:37.024984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966c6c2-5e1f-4f76-9712-2de46abe77c5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:37.031535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500447-f35c-47fa-8fc5-31800d7a9141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:37.032341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500447-f35c-47fa-8fc5-31800d7a9141 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:37.032728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500447-f35c-47fa-8fc5-31800d7a9141 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:37.033691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500447-f35c-47fa-8fc5-31800d7a9141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:37.047903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500447-f35c-47fa-8fc5-31800d7a9141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:37.050053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500447-f35c-47fa-8fc5-31800d7a9141 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:37.063076 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 862/1706] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:37.099222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 845/1707] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:36 2022] POST /identity/v3/projects => generated 355 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:37.100654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500447-f35c-47fa-8fc5-31800d7a9141 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:37.120281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500447-f35c-47fa-8fc5-31800d7a9141 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:37.150757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:37.264778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d11dc1-8452-4eff-b4a8-db7192460083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:37.277250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11dc1-8452-4eff-b4a8-db7192460083 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:37.277596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11dc1-8452-4eff-b4a8-db7192460083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:37.279056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11dc1-8452-4eff-b4a8-db7192460083 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:37.280660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d11dc1-8452-4eff-b4a8-db7192460083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:37.285210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d11dc1-8452-4eff-b4a8-db7192460083 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:37.325883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 863/1708] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:37 2022] POST /identity/v3/users => generated 284 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:37.329838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bdabb-9c53-4226-bf10-42c14821a439 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:37.355589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bdabb-9c53-4226-bf10-42c14821a439 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:37.368231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb658d-3792-4dc5-b1c3-0d4b8b6cbd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:37.375442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb658d-3792-4dc5-b1c3-0d4b8b6cbd4b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:37.375900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb658d-3792-4dc5-b1c3-0d4b8b6cbd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:37.376292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb658d-3792-4dc5-b1c3-0d4b8b6cbd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:37.377255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb658d-3792-4dc5-b1c3-0d4b8b6cbd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:37.381816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb658d-3792-4dc5-b1c3-0d4b8b6cbd4b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:37.439924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d11dc1-8452-4eff-b4a8-db7192460083 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:37.440050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 846/1709] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:25:37.497127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d11dc1-8452-4eff-b4a8-db7192460083 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:37.511040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a5307-2603-44ca-b4bf-9302a9b51b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:37.512824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a5307-2603-44ca-b4bf-9302a9b51b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:37.518343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a5307-2603-44ca-b4bf-9302a9b51b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:37.520228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a5307-2603-44ca-b4bf-9302a9b51b45 admin admin] PATH_INFO: `/v3/projects/dfeeb42a6826496e8b0c08bd499fe41d/users/e3fba92b1eec4e0dbb743b7d82f966fe/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:37.581976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a5307-2603-44ca-b4bf-9302a9b51b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfeeb42a6826496e8b0c08bd499fe41d, user_id=e3fba92b1eec4e0dbb743b7d82f966fe, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:37.595519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a5307-2603-44ca-b4bf-9302a9b51b45 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:37.632443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb658d-3792-4dc5-b1c3-0d4b8b6cbd4b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:37.633591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 864/1710] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:37 2022] PUT /identity/v3/projects/dfeeb42a6826496e8b0c08bd499fe41d/users/e3fba92b1eec4e0dbb743b7d82f966fe/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:37.649420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb658d-3792-4dc5-b1c3-0d4b8b6cbd4b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:37.667071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a5307-2603-44ca-b4bf-9302a9b51b45 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:37.860799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6959c-2174-47fe-94eb-69dc06f6e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:37.863284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6959c-2174-47fe-94eb-69dc06f6e0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:37.863843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6959c-2174-47fe-94eb-69dc06f6e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:37.864331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6959c-2174-47fe-94eb-69dc06f6e0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:37.865998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6959c-2174-47fe-94eb-69dc06f6e0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:37.870639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6959c-2174-47fe-94eb-69dc06f6e0f0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:37.874807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a5307-2603-44ca-b4bf-9302a9b51b45 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:37.891634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:37.900195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:37.901602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:37.903951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] PATH_INFO: `/v3/users/44c72580324a4d90acc9e96209fb7663` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:37.906656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44c72580324a4d90acc9e96209fb7663)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:37.910743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:37.924695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 847/1711] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:37.925616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6959c-2174-47fe-94eb-69dc06f6e0f0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:37.949307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6959c-2174-47fe-94eb-69dc06f6e0f0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:37.955732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2c14-7286-48ad-aa89-90b40c39d08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:37.957047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2c14-7286-48ad-aa89-90b40c39d08e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:37.957341 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2c14-7286-48ad-aa89-90b40c39d08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:37.957977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2c14-7286-48ad-aa89-90b40c39d08e admin admin] PATH_INFO: `/v3/projects/dfeeb42a6826496e8b0c08bd499fe41d/users/e3fba92b1eec4e0dbb743b7d82f966fe/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:37.960546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e2c14-7286-48ad-aa89-90b40c39d08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfeeb42a6826496e8b0c08bd499fe41d, user_id=e3fba92b1eec4e0dbb743b7d82f966fe, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:37.965094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e2c14-7286-48ad-aa89-90b40c39d08e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:38.010193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 848/1712] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:37 2022] PUT /identity/v3/projects/dfeeb42a6826496e8b0c08bd499fe41d/users/e3fba92b1eec4e0dbb743b7d82f966fe/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:38.029138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9f207-a167-447b-812f-1703469cb73b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:38.030266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9f207-a167-447b-812f-1703469cb73b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:38.030266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9f207-a167-447b-812f-1703469cb73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:38.032575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44c72580324a4d90acc9e96209fb7663'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:38.034586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44c72580324a4d90acc9e96209fb7663'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:38.075144 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44c72580324a4d90acc9e96209fb7663'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:38.098615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8af9f207-a167-447b-812f-1703469cb73b None None] MFA Rules not processed for user `e3fba92b1eec4e0dbb743b7d82f966fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:38.099761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8af9f207-a167-447b-812f-1703469cb73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:38.115362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44c72580324a4d90acc9e96209fb7663'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:38.117283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-31002c4f-1ecf-4d37-921e-348ba9bab6d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44c72580324a4d90acc9e96209fb7663'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:38.146837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:38.157316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 865/1713] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:37 2022] DELETE /identity/v3/users/44c72580324a4d90acc9e96209fb7663 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:38.162213 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4086dfd-6914-4fbc-892f-4142ffb4ca9f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:38.169858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4086dfd-6914-4fbc-892f-4142ffb4ca9f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:38.170189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4086dfd-6914-4fbc-892f-4142ffb4ca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:38.212423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d4086dfd-6914-4fbc-892f-4142ffb4ca9f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:38.214947 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4086dfd-6914-4fbc-892f-4142ffb4ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:38.296312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 849/1714] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:38 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:38.303846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f50d7-7c00-4ff9-b26a-941d7e22729b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:38.303846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f50d7-7c00-4ff9-b26a-941d7e22729b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:38.304860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f50d7-7c00-4ff9-b26a-941d7e22729b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:38.357189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ae0f50d7-7c00-4ff9-b26a-941d7e22729b None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:38.359824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ae0f50d7-7c00-4ff9-b26a-941d7e22729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:38.465771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:38.468578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 866/1715] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:38.472296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4086dfd-6914-4fbc-892f-4142ffb4ca9f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:38.477457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4086dfd-6914-4fbc-892f-4142ffb4ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:38.528938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 850/1716] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:25:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:38.569286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f50d7-7c00-4ff9-b26a-941d7e22729b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:38.575621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ae0f50d7-7c00-4ff9-b26a-941d7e22729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:38.632684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4086dfd-6914-4fbc-892f-4142ffb4ca9f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:38.642785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:38.644156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:38.644669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:38.645165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:38.743158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f50d7-7c00-4ff9-b26a-941d7e22729b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:38.753431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741d6ff-8529-46d4-a7d0-5781660199d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:38.754122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d6ff-8529-46d4-a7d0-5781660199d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:38.757972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d6ff-8529-46d4-a7d0-5781660199d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:38.758283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d6ff-8529-46d4-a7d0-5781660199d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:38.780402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:38.781941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:38.786761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741d6ff-8529-46d4-a7d0-5781660199d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:38.788270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741d6ff-8529-46d4-a7d0-5781660199d6 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:38.796048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 867/1717] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:38 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:38.816138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 851/1718] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:38.837338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:38.839252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:38.923559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741d6ff-8529-46d4-a7d0-5781660199d6 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:38.935167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741d6ff-8529-46d4-a7d0-5781660199d6 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:38.956849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:38.957534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:38.957818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:38.958096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:38.961565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:38.975380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:39.030259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717acf5-a1e3-414e-b5fd-90799c8ba3e9 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:39.046912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2dd8a-6a4f-4e50-b7f3-8920d28b0a85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:39.048643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2dd8a-6a4f-4e50-b7f3-8920d28b0a85 service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:39.049069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2dd8a-6a4f-4e50-b7f3-8920d28b0a85 service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:39.049487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2dd8a-6a4f-4e50-b7f3-8920d28b0a85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:39.070663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2dd8a-6a4f-4e50-b7f3-8920d28b0a85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:39.076580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2dd8a-6a4f-4e50-b7f3-8920d28b0a85 service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:39.101306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 868/1719] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:25:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 17:25:39.157901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:39.167748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:39.212579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 852/1720] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:38 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:39.314875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12196-9cde-40ad-bcdc-20b294fd5db2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:39.315233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12196-9cde-40ad-bcdc-20b294fd5db2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:39.315986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12196-9cde-40ad-bcdc-20b294fd5db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:39.381407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8fc12196-9cde-40ad-bcdc-20b294fd5db2 None None] MFA Rules not processed for user `5df621160a0a47ed8a1eb2c9e5645e63`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:39.384513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8fc12196-9cde-40ad-bcdc-20b294fd5db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:39.607423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 869/1721] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:39.612485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:39.614719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:39.637346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67342b4-c5fc-4ac4-850a-034cbe1e018a service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:39.644141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1810e32-ff23-4922-8857-160e90176646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:39.645148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1810e32-ff23-4922-8857-160e90176646 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:39.645645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1810e32-ff23-4922-8857-160e90176646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:39.646260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1810e32-ff23-4922-8857-160e90176646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:39.658906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1810e32-ff23-4922-8857-160e90176646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:39.667421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1810e32-ff23-4922-8857-160e90176646 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:39.796893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc12196-9cde-40ad-bcdc-20b294fd5db2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:39.797637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 853/1722] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:39 2022] POST /identity/v3/projects => generated 403 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:39.837508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc12196-9cde-40ad-bcdc-20b294fd5db2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:40.094811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d50c-d8e8-4943-aad1-e90d4b361d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:40.096736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d50c-d8e8-4943-aad1-e90d4b361d7a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:40.097154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d50c-d8e8-4943-aad1-e90d4b361d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:40.099554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d50c-d8e8-4943-aad1-e90d4b361d7a admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:40.102573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4d50c-d8e8-4943-aad1-e90d4b361d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:40.105780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4d50c-d8e8-4943-aad1-e90d4b361d7a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:40.186226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1810e32-ff23-4922-8857-160e90176646 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:40.196093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 870/1723] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:39 2022] POST /identity/v3/users => generated 308 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:40.203215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1810e32-ff23-4922-8857-160e90176646 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:40.216780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf622-1f24-48d2-bdbb-7baee3e9d049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:40.218044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf622-1f24-48d2-bdbb-7baee3e9d049 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:40.218439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf622-1f24-48d2-bdbb-7baee3e9d049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:40.218827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf622-1f24-48d2-bdbb-7baee3e9d049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:40.220215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaf622-1f24-48d2-bdbb-7baee3e9d049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:40.223469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaf622-1f24-48d2-bdbb-7baee3e9d049 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:40.261876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d50c-d8e8-4943-aad1-e90d4b361d7a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:40.278665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 854/1724] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:40.293885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d50c-d8e8-4943-aad1-e90d4b361d7a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:40.308632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e3cc0-261e-442b-8a49-4e1d73bd2905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:40.346508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e3cc0-261e-442b-8a49-4e1d73bd2905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:40.348589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e3cc0-261e-442b-8a49-4e1d73bd2905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:40.350641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e3cc0-261e-442b-8a49-4e1d73bd2905 admin admin] PATH_INFO: `/v3/projects/388aea65c89742c18bbb7789409de558/users/a87089aeb3c643618a1f7ff28f363091/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:40.387629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e3cc0-261e-442b-8a49-4e1d73bd2905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388aea65c89742c18bbb7789409de558, user_id=a87089aeb3c643618a1f7ff28f363091, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:40.407275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e3cc0-261e-442b-8a49-4e1d73bd2905 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:40.442907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf622-1f24-48d2-bdbb-7baee3e9d049 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:40.443109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 871/1725] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:40 2022] PUT /identity/v3/projects/388aea65c89742c18bbb7789409de558/users/a87089aeb3c643618a1f7ff28f363091/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:40.448533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf622-1f24-48d2-bdbb-7baee3e9d049 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:40.576369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:40.598096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:40.599137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:40.599605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:40.600087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:40.601111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:40.602887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:40.699509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 855/1726] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:40.700328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e3cc0-261e-442b-8a49-4e1d73bd2905 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:40.732560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e3cc0-261e-442b-8a49-4e1d73bd2905 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:40.763545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490ec5f-639f-4e16-a7b2-d796e5ea2d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:40.769164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ec5f-639f-4e16-a7b2-d796e5ea2d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:40.774426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ec5f-639f-4e16-a7b2-d796e5ea2d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:40.775115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ec5f-639f-4e16-a7b2-d796e5ea2d28 admin admin] PATH_INFO: `/v3/projects/388aea65c89742c18bbb7789409de558/users/a87089aeb3c643618a1f7ff28f363091/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:40.782059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490ec5f-639f-4e16-a7b2-d796e5ea2d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388aea65c89742c18bbb7789409de558, user_id=a87089aeb3c643618a1f7ff28f363091, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:40.801770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490ec5f-639f-4e16-a7b2-d796e5ea2d28 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:40.865125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 872/1727] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:40 2022] PUT /identity/v3/projects/388aea65c89742c18bbb7789409de558/users/a87089aeb3c643618a1f7ff28f363091/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:41.128256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:41.132387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:41.157668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e1b33-baa2-480c-8868-d4382af0d7db admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:41.371047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:41.372375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:41.372807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:41.373422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] PATH_INFO: `/v3/users/55f494e6fed84b7485541ef5cd668701` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:41.376175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f494e6fed84b7485541ef5cd668701)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:41.414741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:41.571733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f494e6fed84b7485541ef5cd668701'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:41.586769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f494e6fed84b7485541ef5cd668701'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:41.617236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f494e6fed84b7485541ef5cd668701'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:41.619285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f494e6fed84b7485541ef5cd668701'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:41.637921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f494e6fed84b7485541ef5cd668701'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:41.653433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 856/1728] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:41 2022] DELETE /identity/v3/users/55f494e6fed84b7485541ef5cd668701 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:41.657811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8467d5-76b8-44c5-8d3a-682b5d86ecfd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:41.659085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8467d5-76b8-44c5-8d3a-682b5d86ecfd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:41.659961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8467d5-76b8-44c5-8d3a-682b5d86ecfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:41.697553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ef8467d5-76b8-44c5-8d3a-682b5d86ecfd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:41.701461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ef8467d5-76b8-44c5-8d3a-682b5d86ecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:41.743523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:41.867348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 873/1729] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:41.896094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995111b-db3e-4a73-9641-be1fef538f8a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:41.896379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995111b-db3e-4a73-9641-be1fef538f8a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:41.896689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995111b-db3e-4a73-9641-be1fef538f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:41.897508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:41.900706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:41.973143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e995111b-db3e-4a73-9641-be1fef538f8a None None] MFA Rules not processed for user `927f21ef1ca74ec58f85181728933e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:41.975618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e995111b-db3e-4a73-9641-be1fef538f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:42.050984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1dba6-b00a-4fa3-941e-7e06243afab7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:42.065215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6885d1e-5709-46ce-a22e-e494fa356607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:42.067235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885d1e-5709-46ce-a22e-e494fa356607 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:42.068546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885d1e-5709-46ce-a22e-e494fa356607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:42.068722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885d1e-5709-46ce-a22e-e494fa356607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:42.093777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6885d1e-5709-46ce-a22e-e494fa356607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:42.096672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6885d1e-5709-46ce-a22e-e494fa356607 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:42.133866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 857/1730] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:42.167330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 874/1731] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:42.174353 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583fcba-3002-43ab-8c60-e42923b949d6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:42.174648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583fcba-3002-43ab-8c60-e42923b949d6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:42.174925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583fcba-3002-43ab-8c60-e42923b949d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:42.274050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2583fcba-3002-43ab-8c60-e42923b949d6 None None] MFA Rules not processed for user `927f21ef1ca74ec58f85181728933e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:42.280835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2583fcba-3002-43ab-8c60-e42923b949d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:42.317681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 858/1732] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:42.329237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995111b-db3e-4a73-9641-be1fef538f8a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:42.330734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e995111b-db3e-4a73-9641-be1fef538f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:42.589839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4171ed-713c-4fbb-b7b3-67e86b97b34f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:42.590382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4171ed-713c-4fbb-b7b3-67e86b97b34f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:42.590855 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4171ed-713c-4fbb-b7b3-67e86b97b34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:42.600660 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:42.640272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1e4171ed-713c-4fbb-b7b3-67e86b97b34f None None] MFA Rules not processed for user `71a48b3b4fa94798b158488f08bae743`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:42.644609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1e4171ed-713c-4fbb-b7b3-67e86b97b34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:42.669643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 859/1733] 10.222.0.16 () {58 vars in 1015 bytes} [Sat May 28 17:25:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:42.671124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995111b-db3e-4a73-9641-be1fef538f8a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:42.686176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:42.686946 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:42.687941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:42.689030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:42.763861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:42.763861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:42.772173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4171ed-713c-4fbb-b7b3-67e86b97b34f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:42.780314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 875/1734] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:42.787068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4171ed-713c-4fbb-b7b3-67e86b97b34f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:42.796706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ba6b1-dc46-4eb0-b010-1322a6388878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:42.797901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ba6b1-dc46-4eb0-b010-1322a6388878 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:42.798329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ba6b1-dc46-4eb0-b010-1322a6388878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:42.798745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ba6b1-dc46-4eb0-b010-1322a6388878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:42.826236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ba6b1-dc46-4eb0-b010-1322a6388878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:42.830147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:42.831895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:42.836939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ba6b1-dc46-4eb0-b010-1322a6388878 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:42.883600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229b688-2156-46ec-9d00-db7a9ab38c00 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:42.922643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 860/1735] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:25:42.924304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:42.926067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:42.926067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:42.933138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] PATH_INFO: `/v3/projects/c5549ad0ab514013be5b2334468bef72` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:42.942742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5549ad0ab514013be5b2334468bef72)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:42.952068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:43.130803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:43.177603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5549ad0ab514013be5b2334468bef72', 'request_id': 'req-2811da91-91e8-40c3-8f4c-25176c83b357'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.198669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5549ad0ab514013be5b2334468bef72', 'request_id': 'req-2811da91-91e8-40c3-8f4c-25176c83b357'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.215135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5549ad0ab514013be5b2334468bef72', 'request_id': 'req-2811da91-91e8-40c3-8f4c-25176c83b357'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.233817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] The token cache is being invalidate because project c5549ad0ab514013be5b2334468bef72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:43.234105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.234779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.252730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 876/1736] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:42 2022] DELETE /identity/v3/projects/c5549ad0ab514013be5b2334468bef72 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:25:43.257990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ba6b1-dc46-4eb0-b010-1322a6388878 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:43.281124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ba6b1-dc46-4eb0-b010-1322a6388878 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:43.493264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:43.493264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:43.493264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:43.493264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] PATH_INFO: `/v3/users/56620d1678404e4cb0f0f2c0abf33749` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:43.493760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] RBAC: Authorizing `identity:delete_user(user_id=56620d1678404e4cb0f0f2c0abf33749)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:43.509311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:43.687084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56620d1678404e4cb0f0f2c0abf33749'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.705140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56620d1678404e4cb0f0f2c0abf33749'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.742674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56620d1678404e4cb0f0f2c0abf33749'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.743614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56620d1678404e4cb0f0f2c0abf33749'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.767477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1ee3c3ed-b893-4921-92b3-7f826f95deee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56620d1678404e4cb0f0f2c0abf33749'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:43.781934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 861/1737] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:43 2022] DELETE /identity/v3/users/56620d1678404e4cb0f0f2c0abf33749 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:44.841681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:44.846094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:44.875392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811da91-91e8-40c3-8f4c-25176c83b357 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:45.089805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b032d8-19fe-47a4-a9a1-631dd5eca683 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:45.090362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b032d8-19fe-47a4-a9a1-631dd5eca683 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:45.090362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b032d8-19fe-47a4-a9a1-631dd5eca683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:45.129845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-86b032d8-19fe-47a4-a9a1-631dd5eca683 None None] MFA Rules not processed for user `7fcc62869ac34381a2327be46511137b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:45.131373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-86b032d8-19fe-47a4-a9a1-631dd5eca683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:45.170965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:45.173768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:45.175399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:45.178510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] PATH_INFO: `/v3/projects/2798e3c92658428f951e104bc6458744` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:45.183146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2798e3c92658428f951e104bc6458744)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:45.191192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:45.299197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 862/1738] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:45.403523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2798e3c92658428f951e104bc6458744', 'request_id': 'req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:45.434135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2798e3c92658428f951e104bc6458744', 'request_id': 'req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:45.458269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2798e3c92658428f951e104bc6458744', 'request_id': 'req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:45.484269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] The token cache is being invalidate because project 2798e3c92658428f951e104bc6458744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:45.484565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:45.485785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:45.493565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 877/1739] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:44 2022] DELETE /identity/v3/projects/2798e3c92658428f951e104bc6458744 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:45.498208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336577b2-0721-4f5d-913a-3c10483a26fc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:45.498678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336577b2-0721-4f5d-913a-3c10483a26fc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:45.499143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336577b2-0721-4f5d-913a-3c10483a26fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:45.537027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:45.539229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:45.579252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-336577b2-0721-4f5d-913a-3c10483a26fc None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:45.580750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-336577b2-0721-4f5d-913a-3c10483a26fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:45.599122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:45.699247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:45.764166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7e445-8a2c-4e30-8aac-5ebd2788d030 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:45.773680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:45.774560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:45.774850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:45.775131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] PATH_INFO: `/v3/projects/edb20ec810d44704b0d0cf9e97cd0a9d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:45.782494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb20ec810d44704b0d0cf9e97cd0a9d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:45.785471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:45.845534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 863/1740] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:45.845534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad61ec-eccc-443c-9fc4-e81774af75bb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:45.845534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad61ec-eccc-443c-9fc4-e81774af75bb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:45.845534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad61ec-eccc-443c-9fc4-e81774af75bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:45.913405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ddad61ec-eccc-443c-9fc4-e81774af75bb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:45.914638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ddad61ec-eccc-443c-9fc4-e81774af75bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:45.951860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb20ec810d44704b0d0cf9e97cd0a9d', 'request_id': 'req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:45.982330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb20ec810d44704b0d0cf9e97cd0a9d', 'request_id': 'req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:46.006570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb20ec810d44704b0d0cf9e97cd0a9d', 'request_id': 'req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:46.037170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] The token cache is being invalidate because project edb20ec810d44704b0d0cf9e97cd0a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:46.041765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:46.061052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-90f72cf8-2993-42bf-8fa3-08b403ad9f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:46.063406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 878/1741] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:45 2022] DELETE /identity/v3/projects/edb20ec810d44704b0d0cf9e97cd0a9d => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:46.070525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958514-3620-4d10-8e7e-8d69b8747779 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.070879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958514-3620-4d10-8e7e-8d69b8747779 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.071085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958514-3620-4d10-8e7e-8d69b8747779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.144933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 864/1742] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:46.151275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d87ebb-2928-4d68-a107-e3fd7322d3ff None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.162850 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d87ebb-2928-4d68-a107-e3fd7322d3ff None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.163128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d87ebb-2928-4d68-a107-e3fd7322d3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.165654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ad958514-3620-4d10-8e7e-8d69b8747779 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:46.167239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ad958514-3620-4d10-8e7e-8d69b8747779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:46.207169 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 879/1743] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:46.207981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad958514-3620-4d10-8e7e-8d69b8747779 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:46.227289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a9d87ebb-2928-4d68-a107-e3fd7322d3ff None None] MFA Rules not processed for user `927f21ef1ca74ec58f85181728933e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:46.239696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a9d87ebb-2928-4d68-a107-e3fd7322d3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:46.253677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad958514-3620-4d10-8e7e-8d69b8747779 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:46.269859 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a66fd-e7f4-411e-955a-76e89682f6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:46.270547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a66fd-e7f4-411e-955a-76e89682f6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.270826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a66fd-e7f4-411e-955a-76e89682f6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.271101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a66fd-e7f4-411e-955a-76e89682f6f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.271873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a66fd-e7f4-411e-955a-76e89682f6f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:46.272183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a66fd-e7f4-411e-955a-76e89682f6f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:46.289653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a66fd-e7f4-411e-955a-76e89682f6f6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:46.348295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 880/1744] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:46.351568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7336-30f9-4c5d-a268-3eedad6b2434 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.351863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7336-30f9-4c5d-a268-3eedad6b2434 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.352147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7336-30f9-4c5d-a268-3eedad6b2434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.417084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 865/1745] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:46.419173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf166809-9a22-4265-ba44-d7b7627938a7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.419778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf166809-9a22-4265-ba44-d7b7627938a7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.420270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf166809-9a22-4265-ba44-d7b7627938a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.475096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bf166809-9a22-4265-ba44-d7b7627938a7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:46.482708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bf166809-9a22-4265-ba44-d7b7627938a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:46.491034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e0ca7336-30f9-4c5d-a268-3eedad6b2434 None None] MFA Rules not processed for user `a87089aeb3c643618a1f7ff28f363091`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:46.501699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e0ca7336-30f9-4c5d-a268-3eedad6b2434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:46.517377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 866/1746] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:46.518568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf166809-9a22-4265-ba44-d7b7627938a7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:46.567456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf166809-9a22-4265-ba44-d7b7627938a7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:46.575152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30b4e9-c2ba-40f7-8591-6a7458b2fdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:46.576049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b4e9-c2ba-40f7-8591-6a7458b2fdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.576566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b4e9-c2ba-40f7-8591-6a7458b2fdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.576996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b4e9-c2ba-40f7-8591-6a7458b2fdc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.578409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30b4e9-c2ba-40f7-8591-6a7458b2fdc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:46.582348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30b4e9-c2ba-40f7-8591-6a7458b2fdc0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:46.645431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 867/1747] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:46 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:46.647674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30b4e9-c2ba-40f7-8591-6a7458b2fdc0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:46.705777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 881/1748] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:46 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:46.708884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7336-30f9-4c5d-a268-3eedad6b2434 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:46.751558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7336-30f9-4c5d-a268-3eedad6b2434 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:46.757695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:46.827672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30b4e9-c2ba-40f7-8591-6a7458b2fdc0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:46.843841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:46.844684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.844802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.845068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.845810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:46.846117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:46.849966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:46.850532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:46.894038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 868/1749] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:46.900342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb7cb-ba66-4048-91a1-0b44c3429810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:46.901568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb7cb-ba66-4048-91a1-0b44c3429810 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.902059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb7cb-ba66-4048-91a1-0b44c3429810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.902507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb7cb-ba66-4048-91a1-0b44c3429810 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.903217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bb7cb-ba66-4048-91a1-0b44c3429810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:46.905211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bb7cb-ba66-4048-91a1-0b44c3429810 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:46.905774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:46.911477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:46.966372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51168325-2fc3-41e9-9e6f-35c1a849d588 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:46.976115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 882/1750] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:46 2022] POST /identity/v3/users => generated 286 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:46.977103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb7cb-ba66-4048-91a1-0b44c3429810 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:46.983899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:46.984856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:46.985122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:46.985409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] PATH_INFO: `/v3/users/232a1bcc0b9644abb625e284e117212c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:46.993296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=232a1bcc0b9644abb625e284e117212c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:46.996417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:47.016373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb7cb-ba66-4048-91a1-0b44c3429810 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:47.034385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924431ba-f872-4358-8628-c2fa90c95930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:47.035179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924431ba-f872-4358-8628-c2fa90c95930 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.035484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924431ba-f872-4358-8628-c2fa90c95930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.035826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924431ba-f872-4358-8628-c2fa90c95930 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.036658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924431ba-f872-4358-8628-c2fa90c95930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:47.038614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924431ba-f872-4358-8628-c2fa90c95930 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:47.122974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 883/1751] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:46 2022] POST /identity/v3/projects => generated 393 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:47.124215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924431ba-f872-4358-8628-c2fa90c95930 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:47.127799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '232a1bcc0b9644abb625e284e117212c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:47.162132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '232a1bcc0b9644abb625e284e117212c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:47.210860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '232a1bcc0b9644abb625e284e117212c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:47.213060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '232a1bcc0b9644abb625e284e117212c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:47.223832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '232a1bcc0b9644abb625e284e117212c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:47.234836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 869/1752] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:46 2022] DELETE /identity/v3/users/232a1bcc0b9644abb625e284e117212c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:47.245487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:47.247279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:47.276547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924431ba-f872-4358-8628-c2fa90c95930 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:47.412973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6d995-e48d-4d2f-805d-acc1c2a942b9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:47.446279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:47.452697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.453357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.453877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.461216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:47.464959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:47.472865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af235609-4e31-4885-89c4-1a45b7cfa701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:47.473628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af235609-4e31-4885-89c4-1a45b7cfa701 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.473918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af235609-4e31-4885-89c4-1a45b7cfa701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.474179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af235609-4e31-4885-89c4-1a45b7cfa701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.475062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af235609-4e31-4885-89c4-1a45b7cfa701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:47.476723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af235609-4e31-4885-89c4-1a45b7cfa701 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:47.493047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 884/1753] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:47.494882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bbd71-2bc9-48c6-aa32-8992d9277483 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.495798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bbd71-2bc9-48c6-aa32-8992d9277483 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.496073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bbd71-2bc9-48c6-aa32-8992d9277483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.512331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 870/1754] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:47 2022] POST /identity/v3/users => generated 297 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:47.512331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:47.516247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:47.549003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-057bbd71-2bc9-48c6-aa32-8992d9277483 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:47.550616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7a6dc-76a9-4b92-baa7-fda6e7d49202 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:47.554339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-057bbd71-2bc9-48c6-aa32-8992d9277483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:47.561062 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcb63f-ec3e-48be-9ca0-4c6d7c14bb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:47.564081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcb63f-ec3e-48be-9ca0-4c6d7c14bb4e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.565234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcb63f-ec3e-48be-9ca0-4c6d7c14bb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.566751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcb63f-ec3e-48be-9ca0-4c6d7c14bb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.570155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcb63f-ec3e-48be-9ca0-4c6d7c14bb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:47.572095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcb63f-ec3e-48be-9ca0-4c6d7c14bb4e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:47.593851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 885/1755] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:47.595016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bbd71-2bc9-48c6-aa32-8992d9277483 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:47.626880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 871/1756] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:47.630343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0bcf2-8c49-486b-9765-43a20c433282 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.632146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bbd71-2bc9-48c6-aa32-8992d9277483 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:47.632506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0bcf2-8c49-486b-9765-43a20c433282 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.633379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0bcf2-8c49-486b-9765-43a20c433282 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.649835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b964c-e803-4c42-952a-f6b82dab1a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:47.650540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b964c-e803-4c42-952a-f6b82dab1a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.651386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b964c-e803-4c42-952a-f6b82dab1a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.655959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 872/1757] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:25:47 2022] GET /identity => generated 270 bytes in 28 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:25:47.657982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b964c-e803-4c42-952a-f6b82dab1a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.658467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b964c-e803-4c42-952a-f6b82dab1a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:47.659849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e14dd-aaed-420a-9c9f-bc33691fa44a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.660156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b964c-e803-4c42-952a-f6b82dab1a62 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:47.660615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e14dd-aaed-420a-9c9f-bc33691fa44a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.662539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e14dd-aaed-420a-9c9f-bc33691fa44a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.665615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 873/1758] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 17:25:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:25:47.671996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e14dd-aaed-420a-9c9f-bc33691fa44a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:47.690074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d57e14dd-aaed-420a-9c9f-bc33691fa44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:47.736236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:47.736732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 886/1759] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:47.740794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b964c-e803-4c42-952a-f6b82dab1a62 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:47.790870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b964c-e803-4c42-952a-f6b82dab1a62 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:47.807256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e25ae-8708-4bbd-bd79-de360540c27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:47.808199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e25ae-8708-4bbd-bd79-de360540c27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.808496 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e25ae-8708-4bbd-bd79-de360540c27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.809906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e25ae-8708-4bbd-bd79-de360540c27f admin admin] PATH_INFO: `/v3/projects/20b273d3719040838030bf0984f8ac5a/users/2b5b5818add6408cabf259b9b035ee2d/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.851394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e25ae-8708-4bbd-bd79-de360540c27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b273d3719040838030bf0984f8ac5a, user_id=2b5b5818add6408cabf259b9b035ee2d, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:47.855556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e25ae-8708-4bbd-bd79-de360540c27f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:47.875791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e14dd-aaed-420a-9c9f-bc33691fa44a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:47.895536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 887/1760] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:47 2022] PUT /identity/v3/projects/20b273d3719040838030bf0984f8ac5a/users/2b5b5818add6408cabf259b9b035ee2d/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:47.899087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226d3d1-f972-46c4-a892-0b91e8ddf6a6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:47.900319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226d3d1-f972-46c4-a892-0b91e8ddf6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:47.901087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226d3d1-f972-46c4-a892-0b91e8ddf6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:47.949542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2226d3d1-f972-46c4-a892-0b91e8ddf6a6 None None] MFA Rules not processed for user `71a48b3b4fa94798b158488f08bae743`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:47.951851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2226d3d1-f972-46c4-a892-0b91e8ddf6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:47.974793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:48.004333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b8861-43d4-49ab-82c9-0ff784465039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:48.005020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b8861-43d4-49ab-82c9-0ff784465039 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:48.005311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b8861-43d4-49ab-82c9-0ff784465039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:48.005629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b8861-43d4-49ab-82c9-0ff784465039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:48.053290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 888/1761] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:25:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:48.054589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226d3d1-f972-46c4-a892-0b91e8ddf6a6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:48.146753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b8861-43d4-49ab-82c9-0ff784465039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:48.148249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b8861-43d4-49ab-82c9-0ff784465039 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:48.208688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 874/1762] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:48.210145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483e483-0132-45df-ae08-38fada503e31 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:48.212250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483e483-0132-45df-ae08-38fada503e31 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:48.212549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483e483-0132-45df-ae08-38fada503e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:48.317434 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3483e483-0132-45df-ae08-38fada503e31 None None] MFA Rules not processed for user `2b5b5818add6408cabf259b9b035ee2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:48.326521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3483e483-0132-45df-ae08-38fada503e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:48.329091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226d3d1-f972-46c4-a892-0b91e8ddf6a6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:48.342860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a4398-2093-40f1-81a3-f1a980d408bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:48.346451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a4398-2093-40f1-81a3-f1a980d408bf admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:48.347027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a4398-2093-40f1-81a3-f1a980d408bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:48.347716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a4398-2093-40f1-81a3-f1a980d408bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:48.349831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a4398-2093-40f1-81a3-f1a980d408bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:48.358951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a4398-2093-40f1-81a3-f1a980d408bf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:48.376718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 889/1763] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:48.380233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a4398-2093-40f1-81a3-f1a980d408bf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:48.400385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a4398-2093-40f1-81a3-f1a980d408bf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:48.409529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1006abc-2af5-4988-962a-2d3982b9f5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:48.411895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006abc-2af5-4988-962a-2d3982b9f5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:48.412256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006abc-2af5-4988-962a-2d3982b9f5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:48.412797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006abc-2af5-4988-962a-2d3982b9f5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:48.442215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1006abc-2af5-4988-962a-2d3982b9f5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:48.443947 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1006abc-2af5-4988-962a-2d3982b9f5b6 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:48.475648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 890/1764] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:48.480665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585af74-8c2a-4ed7-bccd-05d3ccc28cda None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:48.481398 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585af74-8c2a-4ed7-bccd-05d3ccc28cda None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:48.481773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585af74-8c2a-4ed7-bccd-05d3ccc28cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:48.507208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 875/1765] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:48.509294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483e483-0132-45df-ae08-38fada503e31 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:48.535933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5585af74-8c2a-4ed7-bccd-05d3ccc28cda None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:48.545769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5585af74-8c2a-4ed7-bccd-05d3ccc28cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:48.573569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483e483-0132-45df-ae08-38fada503e31 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:48.586261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a50150-ae4a-4c33-81bd-cdaa500ccf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:48.587188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a50150-ae4a-4c33-81bd-cdaa500ccf5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:48.591175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a50150-ae4a-4c33-81bd-cdaa500ccf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:48.591175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a50150-ae4a-4c33-81bd-cdaa500ccf5c admin admin] PATH_INFO: `/v3/projects/bd54b60c1f974116ba4158331da53a46/users/bb6a9db3ea5c4e53901db7b445e20504/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:48.595447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:48.635088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a50150-ae4a-4c33-81bd-cdaa500ccf5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd54b60c1f974116ba4158331da53a46, user_id=bb6a9db3ea5c4e53901db7b445e20504, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:48.639643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a50150-ae4a-4c33-81bd-cdaa500ccf5c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:48.683437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 876/1766] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:48 2022] PUT /identity/v3/projects/bd54b60c1f974116ba4158331da53a46/users/bb6a9db3ea5c4e53901db7b445e20504/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:48.685227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e47b3d-2cad-45be-82bb-cd5cf2267d07 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:48.685429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e47b3d-2cad-45be-82bb-cd5cf2267d07 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:48.685713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e47b3d-2cad-45be-82bb-cd5cf2267d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:48.789828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 891/1767] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:25:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:48.795420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-47e47b3d-2cad-45be-82bb-cd5cf2267d07 None None] MFA Rules not processed for user `bb6a9db3ea5c4e53901db7b445e20504`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:48.796683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-47e47b3d-2cad-45be-82bb-cd5cf2267d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:48.951103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 877/1768] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:48.955729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585af74-8c2a-4ed7-bccd-05d3ccc28cda None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:48.989564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585af74-8c2a-4ed7-bccd-05d3ccc28cda None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:49.099494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:49.162565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce23e7c-a4b5-4972-a8b6-bf0632db4f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:49.163245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23e7c-a4b5-4972-a8b6-bf0632db4f03 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:49.163563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23e7c-a4b5-4972-a8b6-bf0632db4f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:49.163843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23e7c-a4b5-4972-a8b6-bf0632db4f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:49.164584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce23e7c-a4b5-4972-a8b6-bf0632db4f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:49.167259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce23e7c-a4b5-4972-a8b6-bf0632db4f03 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:49.226590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 892/1769] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:48 2022] POST /identity/v3/projects => generated 389 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:49.243021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e47b3d-2cad-45be-82bb-cd5cf2267d07 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:49.259819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e47b3d-2cad-45be-82bb-cd5cf2267d07 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:49.403093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609db23-bbb9-4555-8404-dd4285873cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:49.403818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609db23-bbb9-4555-8404-dd4285873cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:49.404099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609db23-bbb9-4555-8404-dd4285873cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:49.404374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609db23-bbb9-4555-8404-dd4285873cd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:49.405075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609db23-bbb9-4555-8404-dd4285873cd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:49.406997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609db23-bbb9-4555-8404-dd4285873cd8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:49.454288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 878/1770] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:49 2022] POST /identity/v3/users => generated 301 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:49.455560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce23e7c-a4b5-4972-a8b6-bf0632db4f03 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:49.474179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce23e7c-a4b5-4972-a8b6-bf0632db4f03 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:49.501439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500aa5-da7a-4973-a969-5f52bc06b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:49.502095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500aa5-da7a-4973-a969-5f52bc06b7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:49.502394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500aa5-da7a-4973-a969-5f52bc06b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:49.502672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500aa5-da7a-4973-a969-5f52bc06b7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:49.503446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500aa5-da7a-4973-a969-5f52bc06b7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:49.514803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500aa5-da7a-4973-a969-5f52bc06b7ff admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:49.575722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 893/1771] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:49.577770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609db23-bbb9-4555-8404-dd4285873cd8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:49.595725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609db23-bbb9-4555-8404-dd4285873cd8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:49.601824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:49.602896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:49.603140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:49.603448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] PATH_INFO: `/v3/projects/f87497e5e41a48b79a09274710437f25/users/1cc8d69264524e9084321faab0186d3c/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:49.644371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87497e5e41a48b79a09274710437f25, user_id=1cc8d69264524e9084321faab0186d3c, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:49.647662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:49.704408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 879/1772] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:49 2022] PUT /identity/v3/projects/f87497e5e41a48b79a09274710437f25/users/1cc8d69264524e9084321faab0186d3c/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:49.705930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500aa5-da7a-4973-a969-5f52bc06b7ff admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:49.734050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:49.737247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:49.746398 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500aa5-da7a-4973-a969-5f52bc06b7ff admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:49.977083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826216b-ccf7-48db-993f-ea777c08424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:49.977626 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:49.977784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826216b-ccf7-48db-993f-ea777c08424a admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:49.979607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826216b-ccf7-48db-993f-ea777c08424a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:49.980338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826216b-ccf7-48db-993f-ea777c08424a admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:49.983096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826216b-ccf7-48db-993f-ea777c08424a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:49.989518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826216b-ccf7-48db-993f-ea777c08424a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:50.082410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 894/1773] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:50.087089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826216b-ccf7-48db-993f-ea777c08424a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:50.121504 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826216b-ccf7-48db-993f-ea777c08424a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:50.122374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6db92-345d-49ca-958a-57215e60b978 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:50.141713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:50.143481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:50.144729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885323cf-5c67-4475-a7f8-3f0140283bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:50.145631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885323cf-5c67-4475-a7f8-3f0140283bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:50.146389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:50.146514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] PATH_INFO: `/v3/projects/6d6dc24eb1bb42c7a348f24ea101b166` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:50.150519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885323cf-5c67-4475-a7f8-3f0140283bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:50.150964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885323cf-5c67-4475-a7f8-3f0140283bc6 admin admin] PATH_INFO: `/v3/projects/f87497e5e41a48b79a09274710437f25/users/1cc8d69264524e9084321faab0186d3c/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:50.152362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6dc24eb1bb42c7a348f24ea101b166)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:50.158591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:50.162728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885323cf-5c67-4475-a7f8-3f0140283bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87497e5e41a48b79a09274710437f25, user_id=1cc8d69264524e9084321faab0186d3c, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:50.175800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885323cf-5c67-4475-a7f8-3f0140283bc6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:50.235109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 895/1774] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:50 2022] PUT /identity/v3/projects/f87497e5e41a48b79a09274710437f25/users/1cc8d69264524e9084321faab0186d3c/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:50.236737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb44f7c-59ae-4320-b581-78a33c59ded3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:50.237033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb44f7c-59ae-4320-b581-78a33c59ded3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:50.237303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb44f7c-59ae-4320-b581-78a33c59ded3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:50.315042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0eb44f7c-59ae-4320-b581-78a33c59ded3 None None] MFA Rules not processed for user `1cc8d69264524e9084321faab0186d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:50.319223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0eb44f7c-59ae-4320-b581-78a33c59ded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:50.320524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6dc24eb1bb42c7a348f24ea101b166', 'request_id': 'req-fc3861d9-8cba-4ada-a551-68117b40cdba'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:50.348927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6dc24eb1bb42c7a348f24ea101b166', 'request_id': 'req-fc3861d9-8cba-4ada-a551-68117b40cdba'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:50.375871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6dc24eb1bb42c7a348f24ea101b166', 'request_id': 'req-fc3861d9-8cba-4ada-a551-68117b40cdba'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:50.423109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] The token cache is being invalidate because project 6d6dc24eb1bb42c7a348f24ea101b166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:50.427027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:50.428593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc3861d9-8cba-4ada-a551-68117b40cdba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:50.440326 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 880/1775] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:49 2022] DELETE /identity/v3/projects/6d6dc24eb1bb42c7a348f24ea101b166 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:50.444684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347b3c1-df56-474a-8093-7672ef69fdaa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:50.447813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347b3c1-df56-474a-8093-7672ef69fdaa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:50.450756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347b3c1-df56-474a-8093-7672ef69fdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:50.519047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0347b3c1-df56-474a-8093-7672ef69fdaa None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:50.524039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0347b3c1-df56-474a-8093-7672ef69fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:50.627086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 896/1776] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:50.648570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb44f7c-59ae-4320-b581-78a33c59ded3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:50.650070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0eb44f7c-59ae-4320-b581-78a33c59ded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:50.706086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 881/1777] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:50.717586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621081fe-77f2-4a14-a69b-a6c61917740c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:50.717900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621081fe-77f2-4a14-a69b-a6c61917740c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:50.718154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621081fe-77f2-4a14-a69b-a6c61917740c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:50.763715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:50.784567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-621081fe-77f2-4a14-a69b-a6c61917740c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:50.789244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-621081fe-77f2-4a14-a69b-a6c61917740c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:50.816353 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 882/1778] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:50.818483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621081fe-77f2-4a14-a69b-a6c61917740c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:50.829206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-621081fe-77f2-4a14-a69b-a6c61917740c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:50.856862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb44f7c-59ae-4320-b581-78a33c59ded3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:50.875269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e67a4-8859-4bf4-97ac-aedd67052aa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:50.875971 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e67a4-8859-4bf4-97ac-aedd67052aa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:50.876250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e67a4-8859-4bf4-97ac-aedd67052aa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:50.876525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e67a4-8859-4bf4-97ac-aedd67052aa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:50.891112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e67a4-8859-4bf4-97ac-aedd67052aa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:50.893259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e67a4-8859-4bf4-97ac-aedd67052aa5 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:50.914078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 897/1779] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:50.915975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e67a4-8859-4bf4-97ac-aedd67052aa5 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:50.927759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e67a4-8859-4bf4-97ac-aedd67052aa5 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:50.934611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:50.935503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:50.935627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:50.935901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:50.936625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:50.936934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:50.938845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:50.973229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 898/1780] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:50.989992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:51.022137 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2709dd-c997-48e7-b0ff-b9ca447ceaf4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:51.035875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:51.050184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:51.056489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:51.056803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:51.057670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:51.059973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:51.061963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:51.100001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621081fe-77f2-4a14-a69b-a6c61917740c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:51.124744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:51.125455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:51.125851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:51.126681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:51.140230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:51.218452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 899/1781] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:50 2022] POST /identity/v3/projects => generated 365 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:51.219525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:51.230074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:51.411586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:51.413392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:51.476276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d8607-00ff-4921-bf45-c5dc24d987f4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:51.490673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:51.491749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:51.492203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:51.492634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:51.499328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 883/1782] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:51.501278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:51.502415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:51.505358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:51.545179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b5e6d-5b4f-4c39-b577-133c8d349083 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:51.683472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:51.688545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:51.710350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 900/1783] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:51.731602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:51.751377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5891379-f0e1-49f4-aa9c-049e658449d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:51.753210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891379-f0e1-49f4-aa9c-049e658449d3 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:51.753500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891379-f0e1-49f4-aa9c-049e658449d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:51.753787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891379-f0e1-49f4-aa9c-049e658449d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:51.760221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a5891379-f0e1-49f4-aa9c-049e658449d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:51.797402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5891379-f0e1-49f4-aa9c-049e658449d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:51.801967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5891379-f0e1-49f4-aa9c-049e658449d3 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:51.855741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 884/1784] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:51.857735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996af94-4f0e-4452-8ffd-adbaee18482a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:51.858023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996af94-4f0e-4452-8ffd-adbaee18482a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:51.858296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996af94-4f0e-4452-8ffd-adbaee18482a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:51.865556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 885/1785] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:25:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:25:51.880610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:51.912958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e32cd-9797-4b38-a2a1-7c7fabb590c8 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:51.922316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb860947-de20-4311-9a0f-6a0975dbdecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:51.923539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb860947-de20-4311-9a0f-6a0975dbdecf admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:51.923980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb860947-de20-4311-9a0f-6a0975dbdecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:51.925155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb860947-de20-4311-9a0f-6a0975dbdecf admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:51.926907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb860947-de20-4311-9a0f-6a0975dbdecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:51.931012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb860947-de20-4311-9a0f-6a0975dbdecf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:51.961654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3deb1a-d6ed-4850-ade6-0d1c4a5e2fe8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:51.962160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3deb1a-d6ed-4850-ade6-0d1c4a5e2fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:51.962584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3deb1a-d6ed-4850-ade6-0d1c4a5e2fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:51.997525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 901/1786] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:51 2022] POST /identity/v3/users => generated 283 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:52.005455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb860947-de20-4311-9a0f-6a0975dbdecf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:52.031091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0d3deb1a-d6ed-4850-ade6-0d1c4a5e2fe8 None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:52.035438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0d3deb1a-d6ed-4850-ade6-0d1c4a5e2fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:52.070614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb860947-de20-4311-9a0f-6a0975dbdecf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:52.102056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149cf1f-cf45-4bbe-adc5-a14aef013f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:52.102921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149cf1f-cf45-4bbe-adc5-a14aef013f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:52.106819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149cf1f-cf45-4bbe-adc5-a14aef013f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:52.106999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149cf1f-cf45-4bbe-adc5-a14aef013f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:52.107953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149cf1f-cf45-4bbe-adc5-a14aef013f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:52.137032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149cf1f-cf45-4bbe-adc5-a14aef013f2c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:52.225285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 902/1787] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:52.230769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149cf1f-cf45-4bbe-adc5-a14aef013f2c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:52.299856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149cf1f-cf45-4bbe-adc5-a14aef013f2c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:52.333737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adecb0-2749-46f6-9c5f-388bee29dccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:52.337408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adecb0-2749-46f6-9c5f-388bee29dccc admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:52.340678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adecb0-2749-46f6-9c5f-388bee29dccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:52.341623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adecb0-2749-46f6-9c5f-388bee29dccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:52.348899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adecb0-2749-46f6-9c5f-388bee29dccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:52.355183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adecb0-2749-46f6-9c5f-388bee29dccc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:52.376938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 886/1788] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:25:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:52.457326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 903/1789] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:52.461682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3deb1a-d6ed-4850-ade6-0d1c4a5e2fe8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:52.480895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3deb1a-d6ed-4850-ade6-0d1c4a5e2fe8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:52.508192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:52.509140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:52.509600 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:52.510612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] PATH_INFO: `/v3/projects/a2d211c2aa104520ac2ba72795bf7af6/users/df05c41569634e81bc2fac1fd1d71d5a/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:52.579442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d211c2aa104520ac2ba72795bf7af6, user_id=df05c41569634e81bc2fac1fd1d71d5a, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:52.582722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:52.602239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:52.636548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 887/1790] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:52 2022] PUT /identity/v3/projects/a2d211c2aa104520ac2ba72795bf7af6/users/df05c41569634e81bc2fac1fd1d71d5a/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:52.646839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efb8b2-ad48-4673-94a0-7a3e9d952fa5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:52.647205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efb8b2-ad48-4673-94a0-7a3e9d952fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:52.647424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efb8b2-ad48-4673-94a0-7a3e9d952fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:52.741646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:52.744666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:52.800505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-50efb8b2-ad48-4673-94a0-7a3e9d952fa5 None None] MFA Rules not processed for user `df05c41569634e81bc2fac1fd1d71d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:52.803118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-50efb8b2-ad48-4673-94a0-7a3e9d952fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:53.062277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9d9cf-170b-4b7e-8cc7-c0a592c72201 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:53.081262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cbe81-f7f1-41df-8ffc-e76004a1a491 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:53.081952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cbe81-f7f1-41df-8ffc-e76004a1a491 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:53.082238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cbe81-f7f1-41df-8ffc-e76004a1a491 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:53.082521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cbe81-f7f1-41df-8ffc-e76004a1a491 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:53.116440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cbe81-f7f1-41df-8ffc-e76004a1a491 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:53.116618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 904/1791] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:53.117347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cbe81-f7f1-41df-8ffc-e76004a1a491 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:53.117486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efb8b2-ad48-4673-94a0-7a3e9d952fa5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:53.157117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 888/1792] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:25:53.281672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efb8b2-ad48-4673-94a0-7a3e9d952fa5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:53.317209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109753-78b6-47de-87f0-d882f8ed74ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:53.318014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109753-78b6-47de-87f0-d882f8ed74ee admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:53.318391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109753-78b6-47de-87f0-d882f8ed74ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:53.320239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109753-78b6-47de-87f0-d882f8ed74ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:53.320239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab109753-78b6-47de-87f0-d882f8ed74ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:53.326228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab109753-78b6-47de-87f0-d882f8ed74ee admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:53.405486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 905/1793] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:53 2022] POST /identity/v3/projects => generated 363 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:53.409182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cbe81-f7f1-41df-8ffc-e76004a1a491 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:53.430947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cbe81-f7f1-41df-8ffc-e76004a1a491 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:53.569154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f9d97-2ed5-4eac-9eca-bd456f3d2f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:53.570754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f9d97-2ed5-4eac-9eca-bd456f3d2f36 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:53.571486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f9d97-2ed5-4eac-9eca-bd456f3d2f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:53.572166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f9d97-2ed5-4eac-9eca-bd456f3d2f36 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:53.573474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f9d97-2ed5-4eac-9eca-bd456f3d2f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:53.576807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f9d97-2ed5-4eac-9eca-bd456f3d2f36 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:53.614364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 889/1794] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:53 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:53.617891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109753-78b6-47de-87f0-d882f8ed74ee admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:53.638580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109753-78b6-47de-87f0-d882f8ed74ee admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:53.653242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61f155-c7a6-45ea-ae2b-68614e15b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:53.654043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f155-c7a6-45ea-ae2b-68614e15b452 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:53.654418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f155-c7a6-45ea-ae2b-68614e15b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:53.654819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f155-c7a6-45ea-ae2b-68614e15b452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:53.655683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61f155-c7a6-45ea-ae2b-68614e15b452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:53.657407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61f155-c7a6-45ea-ae2b-68614e15b452 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:53.698802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 906/1795] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:53.702833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f9d97-2ed5-4eac-9eca-bd456f3d2f36 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:53.715646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f9d97-2ed5-4eac-9eca-bd456f3d2f36 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:53.734966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:53.736085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:53.736449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:53.736735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] PATH_INFO: `/v3/projects/36cff344ec454c43b7901914d39e1460/users/1cc6929c8d4a490f8ae1015f6a707b97/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:53.777778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cff344ec454c43b7901914d39e1460, user_id=1cc6929c8d4a490f8ae1015f6a707b97, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:53.781326 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:53.812182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 890/1796] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:53 2022] PUT /identity/v3/projects/36cff344ec454c43b7901914d39e1460/users/1cc6929c8d4a490f8ae1015f6a707b97/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:53.813833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61f155-c7a6-45ea-ae2b-68614e15b452 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:53.828097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61f155-c7a6-45ea-ae2b-68614e15b452 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:53.886236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:53.893309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:53.897463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:54.028698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:54.056154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80c650-dc30-413a-a539-2333029f507c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:54.059174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e4bce-51ef-4715-baab-ff3e601a3337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:54.060043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e4bce-51ef-4715-baab-ff3e601a3337 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:54.060449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e4bce-51ef-4715-baab-ff3e601a3337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:54.060658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e4bce-51ef-4715-baab-ff3e601a3337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:54.061492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e4bce-51ef-4715-baab-ff3e601a3337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:54.063858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e4bce-51ef-4715-baab-ff3e601a3337 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:54.070522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:54.071452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:54.071741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:54.072027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] PATH_INFO: `/v3/users/5df621160a0a47ed8a1eb2c9e5645e63` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:54.075324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df621160a0a47ed8a1eb2c9e5645e63)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:54.079399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 907/1797] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:54.080383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:54.081449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e4bce-51ef-4715-baab-ff3e601a3337 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:54.100533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e4bce-51ef-4715-baab-ff3e601a3337 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:54.107745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482017c-cdf5-4652-b48a-a9b7f9fd89e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:54.109016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482017c-cdf5-4652-b48a-a9b7f9fd89e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:54.109443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482017c-cdf5-4652-b48a-a9b7f9fd89e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:54.110040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482017c-cdf5-4652-b48a-a9b7f9fd89e1 admin admin] PATH_INFO: `/v3/projects/36cff344ec454c43b7901914d39e1460/users/1cc6929c8d4a490f8ae1015f6a707b97/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:54.122064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482017c-cdf5-4652-b48a-a9b7f9fd89e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cff344ec454c43b7901914d39e1460, user_id=1cc6929c8d4a490f8ae1015f6a707b97, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:54.129661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482017c-cdf5-4652-b48a-a9b7f9fd89e1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:54.147858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 908/1798] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:54 2022] PUT /identity/v3/projects/36cff344ec454c43b7901914d39e1460/users/1cc6929c8d4a490f8ae1015f6a707b97/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:54.163547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd6d9a-8471-4494-84a2-505dd3df201e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:54.164419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd6d9a-8471-4494-84a2-505dd3df201e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:54.167529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd6d9a-8471-4494-84a2-505dd3df201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:54.191766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df621160a0a47ed8a1eb2c9e5645e63'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:54.211686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df621160a0a47ed8a1eb2c9e5645e63'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:54.247236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-31bd6d9a-8471-4494-84a2-505dd3df201e None None] MFA Rules not processed for user `1cc6929c8d4a490f8ae1015f6a707b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:54.249514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-31bd6d9a-8471-4494-84a2-505dd3df201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:54.251623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df621160a0a47ed8a1eb2c9e5645e63'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:54.252514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df621160a0a47ed8a1eb2c9e5645e63'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:54.265916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df621160a0a47ed8a1eb2c9e5645e63'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:54.281973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 891/1799] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:53 2022] DELETE /identity/v3/users/5df621160a0a47ed8a1eb2c9e5645e63 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:25:54.471542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 909/1800] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:54.644511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:54.646498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:54.730355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd6d9a-8471-4494-84a2-505dd3df201e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:54.732349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-31bd6d9a-8471-4494-84a2-505dd3df201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:54.768933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669ced3-b8c9-4880-bf55-24cec14022a3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:54.776568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a6e07-6f72-47a1-a891-effb463f65a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:54.776833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a6e07-6f72-47a1-a891-effb463f65a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:54.777677 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a6e07-6f72-47a1-a891-effb463f65a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:54.778290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a6e07-6f72-47a1-a891-effb463f65a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:54.891392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd6d9a-8471-4494-84a2-505dd3df201e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:54.910553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a6e07-6f72-47a1-a891-effb463f65a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:54.912119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a6e07-6f72-47a1-a891-effb463f65a1 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:54.925655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 892/1801] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:54.934598 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:54.992069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:54.993230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:54.993521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:54.993832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] PATH_INFO: `/v3/projects/23a35593a4f44634b8f07986236611cf` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:54.996263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a35593a4f44634b8f07986236611cf)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:54.998034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:55.098070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a6e07-6f72-47a1-a891-effb463f65a1 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:55.117042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a6e07-6f72-47a1-a891-effb463f65a1 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:55.122473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a35593a4f44634b8f07986236611cf', 'request_id': 'req-46a47339-2a3e-46e3-8496-946a72d607a8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.143099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a35593a4f44634b8f07986236611cf', 'request_id': 'req-46a47339-2a3e-46e3-8496-946a72d607a8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.160066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a35593a4f44634b8f07986236611cf', 'request_id': 'req-46a47339-2a3e-46e3-8496-946a72d607a8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.165611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:55.190171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] The token cache is being invalidate because project 23a35593a4f44634b8f07986236611cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:55.191321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.192448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.195120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 910/1802] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:54 2022] DELETE /identity/v3/projects/23a35593a4f44634b8f07986236611cf => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:55.201033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:55.204233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:55.253425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:55.254096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:55.254529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:55.254976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:55.264213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:55.351007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47339-2a3e-46e3-8496-946a72d607a8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:55.357477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:55.359259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:55.359621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:55.359908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] PATH_INFO: `/v3/users/a87089aeb3c643618a1f7ff28f363091` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:55.361259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87089aeb3c643618a1f7ff28f363091)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:55.363289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:55.368402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:55.370283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:55.381045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 893/1803] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:25:55.437062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87089aeb3c643618a1f7ff28f363091'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.439270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87089aeb3c643618a1f7ff28f363091'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.448939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87089aeb3c643618a1f7ff28f363091'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.455858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:55.466485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87089aeb3c643618a1f7ff28f363091'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.468649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9a6f06b-7b9f-4d4b-9de3-73ca6b9c2985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87089aeb3c643618a1f7ff28f363091'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.483906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 911/1804] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:55 2022] DELETE /identity/v3/users/a87089aeb3c643618a1f7ff28f363091 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:55.701629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:55.732142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad3b1e-14e1-4d26-b471-838d275d7220 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:55.797073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:55.797942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:55.798218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:55.798489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] PATH_INFO: `/v3/projects/388aea65c89742c18bbb7789409de558` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:55.799687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=388aea65c89742c18bbb7789409de558)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:55.801600 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:55.848883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '388aea65c89742c18bbb7789409de558', 'request_id': 'req-662fad4a-2728-4649-a8dc-0362b5b86735'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.857127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '388aea65c89742c18bbb7789409de558', 'request_id': 'req-662fad4a-2728-4649-a8dc-0362b5b86735'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.864035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '388aea65c89742c18bbb7789409de558', 'request_id': 'req-662fad4a-2728-4649-a8dc-0362b5b86735'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.876609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] The token cache is being invalidate because project 388aea65c89742c18bbb7789409de558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:55.876930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.877565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-662fad4a-2728-4649-a8dc-0362b5b86735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:55.879218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 894/1805] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:55 2022] DELETE /identity/v3/projects/388aea65c89742c18bbb7789409de558 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:55.884290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c83cf9-e5e0-498a-8e95-ef1a5b1a6ddb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:55.884654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c83cf9-e5e0-498a-8e95-ef1a5b1a6ddb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:55.885010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c83cf9-e5e0-498a-8e95-ef1a5b1a6ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:55.904070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c7c83cf9-e5e0-498a-8e95-ef1a5b1a6ddb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:55.905373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c7c83cf9-e5e0-498a-8e95-ef1a5b1a6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:55.968617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 912/1806] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:55.976229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1180f-0480-4d7a-ae7a-95b8096cc5e4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:55.976519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1180f-0480-4d7a-ae7a-95b8096cc5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:55.976784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1180f-0480-4d7a-ae7a-95b8096cc5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:55.984879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:56.002477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-65a1180f-0480-4d7a-ae7a-95b8096cc5e4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:56.004993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-65a1180f-0480-4d7a-ae7a-95b8096cc5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:56.015999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 895/1807] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:56.020460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c83cf9-e5e0-498a-8e95-ef1a5b1a6ddb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.032996 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c83cf9-e5e0-498a-8e95-ef1a5b1a6ddb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:56.040441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:56.041908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.042424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.042711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.043509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:56.043999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:56.045667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:56.061087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 913/1808] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:25:56.065307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1180f-0480-4d7a-ae7a-95b8096cc5e4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.078520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1180f-0480-4d7a-ae7a-95b8096cc5e4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:56.083509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f74b6-fcbf-4617-b9ed-dc3a9ab40f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:56.084299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f74b6-fcbf-4617-b9ed-dc3a9ab40f37 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.084863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f74b6-fcbf-4617-b9ed-dc3a9ab40f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.084863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f74b6-fcbf-4617-b9ed-dc3a9ab40f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.085546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f74b6-fcbf-4617-b9ed-dc3a9ab40f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:56.087625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f74b6-fcbf-4617-b9ed-dc3a9ab40f37 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:56.123929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 896/1809] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:56 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:56.125278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.137508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27606fd-0c30-4fc2-8a96-b2aa8c242e92 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:56.210377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf3fa7-430e-4547-99b3-3ec438f49522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:56.211063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3fa7-430e-4547-99b3-3ec438f49522 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.211370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3fa7-430e-4547-99b3-3ec438f49522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.211756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3fa7-430e-4547-99b3-3ec438f49522 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.212437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf3fa7-430e-4547-99b3-3ec438f49522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:56.215709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf3fa7-430e-4547-99b3-3ec438f49522 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:56.242103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 914/1810] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:56 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:56.248568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f74b6-fcbf-4617-b9ed-dc3a9ab40f37 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.263725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f74b6-fcbf-4617-b9ed-dc3a9ab40f37 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:56.272521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33360d-196e-43ae-bfa0-1c0492c497b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:56.273900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33360d-196e-43ae-bfa0-1c0492c497b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.274224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33360d-196e-43ae-bfa0-1c0492c497b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.275088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33360d-196e-43ae-bfa0-1c0492c497b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.276270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33360d-196e-43ae-bfa0-1c0492c497b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:56.278576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33360d-196e-43ae-bfa0-1c0492c497b1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:56.295901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 897/1811] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:56.299717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf3fa7-430e-4547-99b3-3ec438f49522 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.312985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf3fa7-430e-4547-99b3-3ec438f49522 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:56.319373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb9f5d-d818-4d7f-b1d6-df1ff8daae8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:56.320400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb9f5d-d818-4d7f-b1d6-df1ff8daae8e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.320664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb9f5d-d818-4d7f-b1d6-df1ff8daae8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.320966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb9f5d-d818-4d7f-b1d6-df1ff8daae8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.322339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb9f5d-d818-4d7f-b1d6-df1ff8daae8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:56.324563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb9f5d-d818-4d7f-b1d6-df1ff8daae8e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:56.349957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 915/1812] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:56.351606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33360d-196e-43ae-bfa0-1c0492c497b1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.366135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33360d-196e-43ae-bfa0-1c0492c497b1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:56.373010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf80ad8-c7d1-4326-8399-0d44ac00813a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:56.374018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80ad8-c7d1-4326-8399-0d44ac00813a admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.374300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80ad8-c7d1-4326-8399-0d44ac00813a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.374577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80ad8-c7d1-4326-8399-0d44ac00813a admin admin] PATH_INFO: `/v3/projects/c62b14b879914ef18d78381ff77ed5c2/users/5e167442a2274b9c8f4ddc799c15c0ea/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.423261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf80ad8-c7d1-4326-8399-0d44ac00813a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62b14b879914ef18d78381ff77ed5c2, user_id=5e167442a2274b9c8f4ddc799c15c0ea, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:56.426260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf80ad8-c7d1-4326-8399-0d44ac00813a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:56.459423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 898/1813] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:56 2022] PUT /identity/v3/projects/c62b14b879914ef18d78381ff77ed5c2/users/5e167442a2274b9c8f4ddc799c15c0ea/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:56.462128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6f5a-0bdd-48d3-8230-b4538f96e940 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.462414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6f5a-0bdd-48d3-8230-b4538f96e940 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.462690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6f5a-0bdd-48d3-8230-b4538f96e940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.498482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8ddd6f5a-0bdd-48d3-8230-b4538f96e940 None None] MFA Rules not processed for user `5e167442a2274b9c8f4ddc799c15c0ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:56.500272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8ddd6f5a-0bdd-48d3-8230-b4538f96e940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:56.555699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:56.615014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 916/1814] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:56.615123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf80ad8-c7d1-4326-8399-0d44ac00813a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.681670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd6f5a-0bdd-48d3-8230-b4538f96e940 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.684630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8ddd6f5a-0bdd-48d3-8230-b4538f96e940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:56.790118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf80ad8-c7d1-4326-8399-0d44ac00813a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:56.800114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd6f5a-0bdd-48d3-8230-b4538f96e940 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:56.808196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa705726-6b80-4922-8498-4e1844dab66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:56.811864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa705726-6b80-4922-8498-4e1844dab66c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.812167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa705726-6b80-4922-8498-4e1844dab66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.812722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa705726-6b80-4922-8498-4e1844dab66c admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.815364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa705726-6b80-4922-8498-4e1844dab66c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:56.815492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:56.816923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:56.817506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa705726-6b80-4922-8498-4e1844dab66c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:56.819596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:56.820109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] PATH_INFO: `/v3/users/2b5b5818add6408cabf259b9b035ee2d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:56.824900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b5b5818add6408cabf259b9b035ee2d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:56.833155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:56.886243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 899/1815] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:56 2022] POST /identity/v3/projects => generated 373 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:56.889402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa705726-6b80-4922-8498-4e1844dab66c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:56.961469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b5b5818add6408cabf259b9b035ee2d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:56.962507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b5b5818add6408cabf259b9b035ee2d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:56.989135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b5b5818add6408cabf259b9b035ee2d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:57.032268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa705726-6b80-4922-8498-4e1844dab66c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:57.036251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b5b5818add6408cabf259b9b035ee2d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:57.036979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14feec06-696c-480d-afb6-e06f0b4d89f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b5b5818add6408cabf259b9b035ee2d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:57.040632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:57.041661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.042195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.042560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.043707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:57.049440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:57.062616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 917/1816] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:56 2022] DELETE /identity/v3/users/2b5b5818add6408cabf259b9b035ee2d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:57.066892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728cf8-92d9-4f60-af4e-3edf3d4aaba5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.067428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728cf8-92d9-4f60-af4e-3edf3d4aaba5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.067850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728cf8-92d9-4f60-af4e-3edf3d4aaba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.094671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 900/1817] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:56 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:25:57.097271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:57.098630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ea728cf8-92d9-4f60-af4e-3edf3d4aaba5 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:57.100054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ea728cf8-92d9-4f60-af4e-3edf3d4aaba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:57.101946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:57.127371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0c42f-ae4e-47b7-9f4d-be26d4b0d853 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:57.146186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:57.222134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:57.227010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:57.229959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.230756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.232688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.233815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:57.236646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:57.247865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 918/1818] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:57.247865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea728cf8-92d9-4f60-af4e-3edf3d4aaba5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:57.259390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 901/1819] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:57.260715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:57.262096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ea728cf8-92d9-4f60-af4e-3edf3d4aaba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:57.271782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:57.275455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea728cf8-92d9-4f60-af4e-3edf3d4aaba5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:57.282163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:57.283054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.283331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.283756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] PATH_INFO: `/v3/users/bb6a9db3ea5c4e53901db7b445e20504` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.285130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6a9db3ea5c4e53901db7b445e20504)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:57.287231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:57.362309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6a9db3ea5c4e53901db7b445e20504'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:57.362694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6a9db3ea5c4e53901db7b445e20504'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:57.371802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6a9db3ea5c4e53901db7b445e20504'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:57.386968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6a9db3ea5c4e53901db7b445e20504'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:57.400321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6a9db3ea5c4e53901db7b445e20504'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:57.413596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a98374-eb35-4b40-8bf0-617e66a13218 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:57.425557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d49b-ffcf-4ee9-af28-92075e3d1927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:57.427247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d49b-ffcf-4ee9-af28-92075e3d1927 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.427760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d49b-ffcf-4ee9-af28-92075e3d1927 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.428573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d49b-ffcf-4ee9-af28-92075e3d1927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.431212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3ef7d49b-ffcf-4ee9-af28-92075e3d1927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:57.440616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 919/1820] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:57 2022] DELETE /identity/v3/users/bb6a9db3ea5c4e53901db7b445e20504 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:57.440616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:57.443600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:57.470053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d49b-ffcf-4ee9-af28-92075e3d1927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:57.473621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d49b-ffcf-4ee9-af28-92075e3d1927 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:57.511685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 902/1821] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:57.513603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed090b-8292-4b49-9e8c-0523af59a2eb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.514433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed090b-8292-4b49-9e8c-0523af59a2eb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.515594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed090b-8292-4b49-9e8c-0523af59a2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.559055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a8ed090b-8292-4b49-9e8c-0523af59a2eb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:57.561920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a8ed090b-8292-4b49-9e8c-0523af59a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:57.638385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26399dda-5b96-48d2-b27e-8373bdc3bbfc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:57.647021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbefc6-350d-4906-b4b5-a6825cf603d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:57.649556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbefc6-350d-4906-b4b5-a6825cf603d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.649873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbefc6-350d-4906-b4b5-a6825cf603d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.651520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbefc6-350d-4906-b4b5-a6825cf603d2 admin admin] PATH_INFO: `/v3/projects/b411a6745aef4de1bb6f91d7be1206f7/users/e5dc182c11ae4b83872a165f60f49084/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.672569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbefc6-350d-4906-b4b5-a6825cf603d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b411a6745aef4de1bb6f91d7be1206f7, user_id=e5dc182c11ae4b83872a165f60f49084, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:57.676856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbefc6-350d-4906-b4b5-a6825cf603d2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:57.706210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 920/1822] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:57 2022] PUT /identity/v3/projects/b411a6745aef4de1bb6f91d7be1206f7/users/e5dc182c11ae4b83872a165f60f49084/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:57.710293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbefc6-350d-4906-b4b5-a6825cf603d2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:57.778409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 903/1823] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:57.824692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:57.848633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbefc6-350d-4906-b4b5-a6825cf603d2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:57.856491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed090b-8292-4b49-9e8c-0523af59a2eb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:57.857893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a8ed090b-8292-4b49-9e8c-0523af59a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:57.873657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ca2af-00a2-4a58-b3d0-00eb5e5dcce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:57.874693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ca2af-00a2-4a58-b3d0-00eb5e5dcce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.875893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ca2af-00a2-4a58-b3d0-00eb5e5dcce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.876988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ca2af-00a2-4a58-b3d0-00eb5e5dcce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.881647 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ca2af-00a2-4a58-b3d0-00eb5e5dcce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:57.884863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ca2af-00a2-4a58-b3d0-00eb5e5dcce6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:57.909218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 921/1824] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:25:57.912244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ca2af-00a2-4a58-b3d0-00eb5e5dcce6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:57.941219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ca2af-00a2-4a58-b3d0-00eb5e5dcce6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:57.949975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465e616-be9d-4a55-9057-f959cb9bda33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:57.951314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e616-be9d-4a55-9057-f959cb9bda33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.951973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e616-be9d-4a55-9057-f959cb9bda33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.952276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e616-be9d-4a55-9057-f959cb9bda33 admin admin] PATH_INFO: `/v3/projects/b411a6745aef4de1bb6f91d7be1206f7/users/e5dc182c11ae4b83872a165f60f49084/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:57.955478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465e616-be9d-4a55-9057-f959cb9bda33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b411a6745aef4de1bb6f91d7be1206f7, user_id=e5dc182c11ae4b83872a165f60f49084, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:57.958433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465e616-be9d-4a55-9057-f959cb9bda33 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:57.974016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed090b-8292-4b49-9e8c-0523af59a2eb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:57.987617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 922/1825] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:25:57 2022] PUT /identity/v3/projects/b411a6745aef4de1bb6f91d7be1206f7/users/e5dc182c11ae4b83872a165f60f49084/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:57.989190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:57.991111 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9416972-82dc-42b8-9839-7ff65500ab87 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:57.991585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9416972-82dc-42b8-9839-7ff65500ab87 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:57.991977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9416972-82dc-42b8-9839-7ff65500ab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:58.058647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:58.059604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:58.059986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:58.060361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:58.081330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e9416972-82dc-42b8-9839-7ff65500ab87 None None] MFA Rules not processed for user `e5dc182c11ae4b83872a165f60f49084`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:58.082567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e9416972-82dc-42b8-9839-7ff65500ab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:58.184375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:58.188346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:58.210184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 904/1826] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:58.243153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 923/1827] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:58.258734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:58.261192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:58.331806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d311b6-47f7-48d3-9536-a507cc836626 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:58.336204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:58.336470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:58.336742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:58.337336 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:58.349562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:58.350726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:58.374399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 905/1828] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:25:58.466540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9416972-82dc-42b8-9839-7ff65500ab87 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:58.477143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9416972-82dc-42b8-9839-7ff65500ab87 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:58.481108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a578c-8ccc-495c-8f65-9711d0d2ce10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:58.482032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a578c-8ccc-495c-8f65-9711d0d2ce10 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:58.482308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a578c-8ccc-495c-8f65-9711d0d2ce10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:58.482582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a578c-8ccc-495c-8f65-9711d0d2ce10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:58.484473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-319a578c-8ccc-495c-8f65-9711d0d2ce10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:58.604757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a578c-8ccc-495c-8f65-9711d0d2ce10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:58.612316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a578c-8ccc-495c-8f65-9711d0d2ce10 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:58.649004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 924/1829] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:25:58.828397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:58.830587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:58.861771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c97ec3-a9f5-45d0-a390-c7e8f292ce9f service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:58.879622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:58.880768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:58.881054 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:58.883747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] PATH_INFO: `/v3/projects/20b273d3719040838030bf0984f8ac5a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:58.895661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b273d3719040838030bf0984f8ac5a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:58.899621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:59.024357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b273d3719040838030bf0984f8ac5a', 'request_id': 'req-c9901c5f-f054-4e29-b370-50e049d73ad2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.040132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b273d3719040838030bf0984f8ac5a', 'request_id': 'req-c9901c5f-f054-4e29-b370-50e049d73ad2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.042954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:59.049526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b273d3719040838030bf0984f8ac5a', 'request_id': 'req-c9901c5f-f054-4e29-b370-50e049d73ad2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.084227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] The token cache is being invalidate because project 20b273d3719040838030bf0984f8ac5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:25:59.086298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.087575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.095997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 906/1830] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:58 2022] DELETE /identity/v3/projects/20b273d3719040838030bf0984f8ac5a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:59.097295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a28a3-6198-4939-9cfe-0762f6aed404 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:59.097779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a28a3-6198-4939-9cfe-0762f6aed404 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:59.098306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a28a3-6198-4939-9cfe-0762f6aed404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:59.147373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-eb6a28a3-6198-4939-9cfe-0762f6aed404 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:59.154909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eb6a28a3-6198-4939-9cfe-0762f6aed404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:59.248122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:59.250536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:59.269873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:59.301841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 925/1831] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:59.310919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b88d7f-af47-49be-9a81-945d4087a2c5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:59.311315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b88d7f-af47-49be-9a81-945d4087a2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:59.311684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b88d7f-af47-49be-9a81-945d4087a2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:59.358097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c5b88d7f-af47-49be-9a81-945d4087a2c5 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:59.359368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c5b88d7f-af47-49be-9a81-945d4087a2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:59.374594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 926/1832] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:25:59.378419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b88d7f-af47-49be-9a81-945d4087a2c5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:59.392495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b88d7f-af47-49be-9a81-945d4087a2c5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:59.415732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:59.417453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:59.417780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:59.418064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:59.419943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:25:59.419943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:59.421187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:59.438732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 927/1833] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:25:59.443112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:59.456396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9901c5f-f054-4e29-b370-50e049d73ad2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:59.456396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:59.456396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:59.456396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:59.456722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] PATH_INFO: `/v3/users/df05c41569634e81bc2fac1fd1d71d5a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:59.456722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df05c41569634e81bc2fac1fd1d71d5a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:59.459441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:59.469535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5be14-45b0-4530-8551-5a6de36f66df admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:59.482317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daddea3-3341-42ac-a441-25c2dee94175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:59.483621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daddea3-3341-42ac-a441-25c2dee94175 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:59.484100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daddea3-3341-42ac-a441-25c2dee94175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:59.484573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daddea3-3341-42ac-a441-25c2dee94175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:59.485509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daddea3-3341-42ac-a441-25c2dee94175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:59.487591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daddea3-3341-42ac-a441-25c2dee94175 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:59.537698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 928/1834] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:25:59 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:25:59.540585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df05c41569634e81bc2fac1fd1d71d5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.541763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daddea3-3341-42ac-a441-25c2dee94175 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:59.554344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df05c41569634e81bc2fac1fd1d71d5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.595778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df05c41569634e81bc2fac1fd1d71d5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.597525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df05c41569634e81bc2fac1fd1d71d5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.625490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2d40915c-8422-466f-879f-1317a95579b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df05c41569634e81bc2fac1fd1d71d5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:25:59.640170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 907/1835] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:25:59 2022] DELETE /identity/v3/users/df05c41569634e81bc2fac1fd1d71d5a => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:25:59.644606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532aa4-083a-4146-ad79-a32de77d9520 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:59.645015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532aa4-083a-4146-ad79-a32de77d9520 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:59.645396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532aa4-083a-4146-ad79-a32de77d9520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:59.694586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d3532aa4-083a-4146-ad79-a32de77d9520 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:25:59.699205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d3532aa4-083a-4146-ad79-a32de77d9520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:59.741097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daddea3-3341-42ac-a441-25c2dee94175 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:59.840145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:59.840844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:59.841129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:59.841409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:59.842288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:59.843435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 908/1836] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:25:59.844749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3532aa4-083a-4146-ad79-a32de77d9520 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:59.846345 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:59.846480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d3532aa4-083a-4146-ad79-a32de77d9520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:59.864510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3532aa4-083a-4146-ad79-a32de77d9520 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:25:59.872395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:25:59.873293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:25:59.873806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:25:59.874092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] PATH_INFO: `/v3/projects/bd54b60c1f974116ba4158331da53a46` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:25:59.875922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd54b60c1f974116ba4158331da53a46)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:25:59.878242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:25:59.880480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:25:59.902505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 929/1837] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:25:59 2022] POST /identity/v3/users => generated 285 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:25:59.905497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:25:59.918428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:25:59.996796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd54b60c1f974116ba4158331da53a46', 'request_id': 'req-cc277407-89cf-4519-9187-1f0b321040c5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.013693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd54b60c1f974116ba4158331da53a46', 'request_id': 'req-cc277407-89cf-4519-9187-1f0b321040c5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.027143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd54b60c1f974116ba4158331da53a46', 'request_id': 'req-cc277407-89cf-4519-9187-1f0b321040c5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.059009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] The token cache is being invalidate because project bd54b60c1f974116ba4158331da53a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:00.059472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.062150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.063891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 909/1838] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:25:59 2022] DELETE /identity/v3/projects/bd54b60c1f974116ba4158331da53a46 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:00.081737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:00.088404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:00.103373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab621020-77ed-4c4e-9fd8-2e0715b2ffe8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:00.110397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:00.234581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:00.237018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:00.237753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:00.238662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:00.245080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:00.271936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc277407-89cf-4519-9187-1f0b321040c5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:00.281820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:00.284010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:00.289484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2e9f4-c337-4828-8e47-4324377724b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:00.291534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2e9f4-c337-4828-8e47-4324377724b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:00.292904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2e9f4-c337-4828-8e47-4324377724b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:00.295206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2e9f4-c337-4828-8e47-4324377724b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:00.299552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2e9f4-c337-4828-8e47-4324377724b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:00.301872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2e9f4-c337-4828-8e47-4324377724b9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:00.311936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 930/1839] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:00.312863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:00.314633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:00.321138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 910/1840] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:00.323199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2e9f4-c337-4828-8e47-4324377724b9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:00.337413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2e9f4-c337-4828-8e47-4324377724b9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:00.344205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d802ce-a4a5-4378-94af-79c2bdffe7d6 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:00.344482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ea939-ff74-4fc9-97a0-595d6780ebe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:00.346025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ea939-ff74-4fc9-97a0-595d6780ebe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:00.346961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ea939-ff74-4fc9-97a0-595d6780ebe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:00.347865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ea939-ff74-4fc9-97a0-595d6780ebe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:00.348924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ea939-ff74-4fc9-97a0-595d6780ebe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:00.352739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ea939-ff74-4fc9-97a0-595d6780ebe6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:00.354519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:00.355657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:00.355876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:00.356205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] PATH_INFO: `/v3/users/1cc8d69264524e9084321faab0186d3c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:00.358073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc8d69264524e9084321faab0186d3c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:00.360582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:00.382038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 911/1841] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:00.383724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ea939-ff74-4fc9-97a0-595d6780ebe6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:00.401085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ea939-ff74-4fc9-97a0-595d6780ebe6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:00.408724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddc1e3-95bc-4c74-9e2a-9af8c3c9ee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:00.412828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddc1e3-95bc-4c74-9e2a-9af8c3c9ee43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:00.413471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddc1e3-95bc-4c74-9e2a-9af8c3c9ee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:00.414102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddc1e3-95bc-4c74-9e2a-9af8c3c9ee43 admin admin] PATH_INFO: `/v3/projects/61a01f2f8eff4bc99e8383d1e75abf57/users/6ef8ff51c9eb459aa3b138053224a9d9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:00.439526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddc1e3-95bc-4c74-9e2a-9af8c3c9ee43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a01f2f8eff4bc99e8383d1e75abf57, user_id=6ef8ff51c9eb459aa3b138053224a9d9, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:00.442873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddc1e3-95bc-4c74-9e2a-9af8c3c9ee43 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:00.446870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc8d69264524e9084321faab0186d3c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.449389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc8d69264524e9084321faab0186d3c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.461022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc8d69264524e9084321faab0186d3c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.491121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc8d69264524e9084321faab0186d3c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.494212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc8d69264524e9084321faab0186d3c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:00.498580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 912/1842] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:00 2022] PUT /identity/v3/projects/61a01f2f8eff4bc99e8383d1e75abf57/users/6ef8ff51c9eb459aa3b138053224a9d9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:00.505918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee6ffc-54f4-4d4e-84bc-adeb5092925d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:00.506378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee6ffc-54f4-4d4e-84bc-adeb5092925d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:00.506961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee6ffc-54f4-4d4e-84bc-adeb5092925d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:00.531965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 931/1843] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:00 2022] DELETE /identity/v3/users/1cc8d69264524e9084321faab0186d3c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:00.581094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ebee6ffc-54f4-4d4e-84bc-adeb5092925d None None] MFA Rules not processed for user `6ef8ff51c9eb459aa3b138053224a9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:00.582416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebee6ffc-54f4-4d4e-84bc-adeb5092925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:00.680402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:00.762555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 913/1844] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:00.776066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:00.781077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:00.897143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a2005-839a-4374-ac2b-64c94cf7839b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:00.904226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217f294-6ec6-40f0-868a-c514eef3636c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:00.904565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217f294-6ec6-40f0-868a-c514eef3636c service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:00.904905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217f294-6ec6-40f0-868a-c514eef3636c service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:00.905189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217f294-6ec6-40f0-868a-c514eef3636c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:00.919279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217f294-6ec6-40f0-868a-c514eef3636c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:00.921330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217f294-6ec6-40f0-868a-c514eef3636c service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:00.945490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 932/1845] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:00.982654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee6ffc-54f4-4d4e-84bc-adeb5092925d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:00.984204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebee6ffc-54f4-4d4e-84bc-adeb5092925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:01.134489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee6ffc-54f4-4d4e-84bc-adeb5092925d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:01.142358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:01.145524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:01.145998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:01.146763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] PATH_INFO: `/v3/users/6ef8ff51c9eb459aa3b138053224a9d9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:01.148717 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef8ff51c9eb459aa3b138053224a9d9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:01.155933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:01.293721 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef8ff51c9eb459aa3b138053224a9d9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.312000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef8ff51c9eb459aa3b138053224a9d9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.344997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef8ff51c9eb459aa3b138053224a9d9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.346845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef8ff51c9eb459aa3b138053224a9d9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.371489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef8ff51c9eb459aa3b138053224a9d9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.387751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 914/1846] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:00 2022] DELETE /identity/v3/users/6ef8ff51c9eb459aa3b138053224a9d9 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:01.389307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee3467-02c1-40a5-816a-f5c981f4dacc None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:01.389610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee3467-02c1-40a5-816a-f5c981f4dacc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:01.389893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee3467-02c1-40a5-816a-f5c981f4dacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:01.424558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cdee3467-02c1-40a5-816a-f5c981f4dacc None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:01.428138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cdee3467-02c1-40a5-816a-f5c981f4dacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:01.471676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:01.497466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:01.499056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:01.584907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 933/1847] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:01.601349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee3467-02c1-40a5-816a-f5c981f4dacc None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:01.604434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cdee3467-02c1-40a5-816a-f5c981f4dacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:01.618057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db69b35-8fde-4d32-854a-9d8e534999b3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:01.624401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:01.625666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:01.626074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:01.626391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] PATH_INFO: `/v3/projects/a2d211c2aa104520ac2ba72795bf7af6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:01.627876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d211c2aa104520ac2ba72795bf7af6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:01.630309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:01.632469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:01.723975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d211c2aa104520ac2ba72795bf7af6', 'request_id': 'req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.736777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d211c2aa104520ac2ba72795bf7af6', 'request_id': 'req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.739223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee3467-02c1-40a5-816a-f5c981f4dacc None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:01.749171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d211c2aa104520ac2ba72795bf7af6', 'request_id': 'req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.763621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] The token cache is being invalidate because project a2d211c2aa104520ac2ba72795bf7af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:01.763919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.764603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.766557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 915/1848] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:01 2022] DELETE /identity/v3/projects/a2d211c2aa104520ac2ba72795bf7af6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:01.767647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:01.769573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:01.838914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:01.839770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:01.840053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:01.840331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:01.842172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:01.855931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd57bb8-4ba0-4efb-bbf7-d32a4d84d3cf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:01.861098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:01.862646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:01.862935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:01.863236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] PATH_INFO: `/v3/projects/f87497e5e41a48b79a09274710437f25` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:01.864550 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87497e5e41a48b79a09274710437f25)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:01.866538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:01.917777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:01.919226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:01.921522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87497e5e41a48b79a09274710437f25', 'request_id': 'req-62c420c2-6cab-466b-978d-9f99463f02d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.929114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87497e5e41a48b79a09274710437f25', 'request_id': 'req-62c420c2-6cab-466b-978d-9f99463f02d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.929804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 934/1849] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:01.930781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:01.932824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:01.945657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5eee44-70ab-4484-a45e-7e642ea3ecad service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:01.946827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87497e5e41a48b79a09274710437f25', 'request_id': 'req-62c420c2-6cab-466b-978d-9f99463f02d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.968420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] The token cache is being invalidate because project f87497e5e41a48b79a09274710437f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:01.968817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.970310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-62c420c2-6cab-466b-978d-9f99463f02d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:01.973945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 916/1850] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:01 2022] DELETE /identity/v3/projects/f87497e5e41a48b79a09274710437f25 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:01.975450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c0206-dd49-4f3e-b4c2-0f01a4883906 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:01.975803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c0206-dd49-4f3e-b4c2-0f01a4883906 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:01.976090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c0206-dd49-4f3e-b4c2-0f01a4883906 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:01.977828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 917/1851] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:26:01.983581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:01.983886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c0206-dd49-4f3e-b4c2-0f01a4883906 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:01.988940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bc2c0206-dd49-4f3e-b4c2-0f01a4883906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.039891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:02.039891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.039891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.039891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] PATH_INFO: `/v3/users/108fa4a2eb7f469fa152c5d0ca80e927` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.040557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:02.040694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=108fa4a2eb7f469fa152c5d0ca80e927)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:02.040694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:02.107021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c0206-dd49-4f3e-b4c2-0f01a4883906 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:02.108938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108fa4a2eb7f469fa152c5d0ca80e927'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.109657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108fa4a2eb7f469fa152c5d0ca80e927'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.113272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:02.114750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.115270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.115820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] PATH_INFO: `/v3/users/1cc6929c8d4a490f8ae1015f6a707b97` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.117411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc6929c8d4a490f8ae1015f6a707b97)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:02.121081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:02.121562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108fa4a2eb7f469fa152c5d0ca80e927'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.143262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108fa4a2eb7f469fa152c5d0ca80e927'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.144088 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-590982ba-4f85-4027-a40b-f57447cd5611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108fa4a2eb7f469fa152c5d0ca80e927'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.165779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 935/1852] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:01 2022] DELETE /identity/v3/users/108fa4a2eb7f469fa152c5d0ca80e927 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:02.171134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea255c3-478e-4540-ab0e-2afbc7a72205 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.171533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea255c3-478e-4540-ab0e-2afbc7a72205 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.171887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea255c3-478e-4540-ab0e-2afbc7a72205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.194679 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bea255c3-478e-4540-ab0e-2afbc7a72205 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:02.195802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bea255c3-478e-4540-ab0e-2afbc7a72205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.211492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc6929c8d4a490f8ae1015f6a707b97'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.225123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc6929c8d4a490f8ae1015f6a707b97'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.245835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc6929c8d4a490f8ae1015f6a707b97'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.247823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc6929c8d4a490f8ae1015f6a707b97'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.260398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41fbbb40-1c3a-448d-9e64-8d281157b236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc6929c8d4a490f8ae1015f6a707b97'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.271687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 918/1853] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:01 2022] DELETE /identity/v3/users/1cc6929c8d4a490f8ae1015f6a707b97 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:02.273864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73712256-813e-4352-b56c-66f9cf03de19 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.275936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73712256-813e-4352-b56c-66f9cf03de19 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.276233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73712256-813e-4352-b56c-66f9cf03de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.318929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-73712256-813e-4352-b56c-66f9cf03de19 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:02.322155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-73712256-813e-4352-b56c-66f9cf03de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.368194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 936/1854] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:02.371077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490255b-0424-4945-8eb4-1b75871e0372 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.371580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490255b-0424-4945-8eb4-1b75871e0372 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.371992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490255b-0424-4945-8eb4-1b75871e0372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.413873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4490255b-0424-4945-8eb4-1b75871e0372 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:02.418915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4490255b-0424-4945-8eb4-1b75871e0372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.452573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 919/1855] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:02.454029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73712256-813e-4352-b56c-66f9cf03de19 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:02.456025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-73712256-813e-4352-b56c-66f9cf03de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.499660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:02.531027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 937/1856] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:02.537513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf8f7-d783-4572-bd51-8277a14478ab None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.537741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf8f7-d783-4572-bd51-8277a14478ab None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.538020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf8f7-d783-4572-bd51-8277a14478ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.555514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:02.573973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bffbf8f7-d783-4572-bd51-8277a14478ab None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:02.575618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bffbf8f7-d783-4572-bd51-8277a14478ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.590831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 938/1857] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:02.591772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf8f7-d783-4572-bd51-8277a14478ab None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:02.593238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bffbf8f7-d783-4572-bd51-8277a14478ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.612010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf8f7-d783-4572-bd51-8277a14478ab None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:02.612770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73712256-813e-4352-b56c-66f9cf03de19 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:02.618264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:02.618423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:02.618934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.619218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.619882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.620070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.620220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.620621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] PATH_INFO: `/v3/projects/61a01f2f8eff4bc99e8383d1e75abf57` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.622305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a01f2f8eff4bc99e8383d1e75abf57)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:02.624009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:02.632483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:02.635715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:02.649227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 939/1858] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:02.650079 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:02.652118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.692659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a01f2f8eff4bc99e8383d1e75abf57', 'request_id': 'req-9e54d8c3-f7eb-42d8-8549-2508a4faf697'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.705181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a01f2f8eff4bc99e8383d1e75abf57', 'request_id': 'req-9e54d8c3-f7eb-42d8-8549-2508a4faf697'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.727105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a01f2f8eff4bc99e8383d1e75abf57', 'request_id': 'req-9e54d8c3-f7eb-42d8-8549-2508a4faf697'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.747580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] The token cache is being invalidate because project 61a01f2f8eff4bc99e8383d1e75abf57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:02.748132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.749014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e54d8c3-f7eb-42d8-8549-2508a4faf697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:02.751495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 920/1859] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:02 2022] DELETE /identity/v3/projects/61a01f2f8eff4bc99e8383d1e75abf57 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:02.753614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c21b15-66b3-4b0b-b2a2-1d94b14b6e45 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.754060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c21b15-66b3-4b0b-b2a2-1d94b14b6e45 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.754769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c21b15-66b3-4b0b-b2a2-1d94b14b6e45 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.756587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 921/1860] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:26:02.757395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c21b15-66b3-4b0b-b2a2-1d94b14b6e45 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:02.759573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-05c21b15-66b3-4b0b-b2a2-1d94b14b6e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.762898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b352e3-a283-42fe-a3bc-7c17229a6bc6 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:02.867661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c21b15-66b3-4b0b-b2a2-1d94b14b6e45 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:02.873194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec04d8-0d34-4ff1-a9e0-6d41e08e19ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:02.873949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec04d8-0d34-4ff1-a9e0-6d41e08e19ef admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.874306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec04d8-0d34-4ff1-a9e0-6d41e08e19ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.874667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec04d8-0d34-4ff1-a9e0-6d41e08e19ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.875516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec04d8-0d34-4ff1-a9e0-6d41e08e19ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:02.875915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec04d8-0d34-4ff1-a9e0-6d41e08e19ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:02.877788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec04d8-0d34-4ff1-a9e0-6d41e08e19ef admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:02.895370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 922/1861] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:02.895370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463731c2-921c-4087-884b-9af0481719ac None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.895370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463731c2-921c-4087-884b-9af0481719ac None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.895370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463731c2-921c-4087-884b-9af0481719ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.901986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:02.902894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.903250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.903924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.907136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.917397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-463731c2-921c-4087-884b-9af0481719ac None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:02.918851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-463731c2-921c-4087-884b-9af0481719ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.920735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:02.922257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:02.928829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 923/1862] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:02.930385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522ff3-04e7-4c34-a9d9-05585feba1e7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.930681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522ff3-04e7-4c34-a9d9-05585feba1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.930963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522ff3-04e7-4c34-a9d9-05585feba1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.938785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 940/1863] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:02.940347 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:02.943001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.956192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-62522ff3-04e7-4c34-a9d9-05585feba1e7 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:02.957798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd22bc-345a-4172-9572-b198e1d6e915 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:02.958064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-62522ff3-04e7-4c34-a9d9-05585feba1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:02.962984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:02.964600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:02.964850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:02.965133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] PATH_INFO: `/v3/projects/36cff344ec454c43b7901914d39e1460` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:02.966358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36cff344ec454c43b7901914d39e1460)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:02.968490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.031249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:03.034074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36cff344ec454c43b7901914d39e1460', 'request_id': 'req-52f04335-8b08-4ade-8c5a-7703be6f6196'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.043493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36cff344ec454c43b7901914d39e1460', 'request_id': 'req-52f04335-8b08-4ade-8c5a-7703be6f6196'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.050285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36cff344ec454c43b7901914d39e1460', 'request_id': 'req-52f04335-8b08-4ade-8c5a-7703be6f6196'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.065559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] The token cache is being invalidate because project 36cff344ec454c43b7901914d39e1460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:03.065850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.066530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.068475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 941/1864] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:02 2022] DELETE /identity/v3/projects/36cff344ec454c43b7901914d39e1460 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:03.069012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.070332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.077831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 924/1865] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:03.079776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41bd1-4b41-43b0-b051-e0ebd883700e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.080062 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41bd1-4b41-43b0-b051-e0ebd883700e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.080337 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41bd1-4b41-43b0-b051-e0ebd883700e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.095457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:03.099110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-62e41bd1-4b41-43b0-b051-e0ebd883700e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:03.100425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-62e41bd1-4b41-43b0-b051-e0ebd883700e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.152474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f04335-8b08-4ade-8c5a-7703be6f6196 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.158210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.159817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.160088 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.160367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.161070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.163044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.179082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 925/1866] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:03.180350 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc84d8-cc11-4483-9597-34bb205b6188 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.180710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc84d8-cc11-4483-9597-34bb205b6188 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.181073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc84d8-cc11-4483-9597-34bb205b6188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.195854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 942/1867] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/projects => generated 409 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:03.195854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.198485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.204250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-91dc84d8-cc11-4483-9597-34bb205b6188 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:03.205558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-91dc84d8-cc11-4483-9597-34bb205b6188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.273339 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 926/1868] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:03.274413 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc84d8-cc11-4483-9597-34bb205b6188 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.286143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc84d8-cc11-4483-9597-34bb205b6188 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.287441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e3161-3be8-46bc-b5b0-2b65505ac554 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.294405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884099e-1244-4948-aaf9-937d8c293a44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.294511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.294603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884099e-1244-4948-aaf9-937d8c293a44 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.294890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884099e-1244-4948-aaf9-937d8c293a44 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.295168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884099e-1244-4948-aaf9-937d8c293a44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.295265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.295647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.295935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.296670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:03.296987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.298809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.307514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884099e-1244-4948-aaf9-937d8c293a44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.309480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884099e-1244-4948-aaf9-937d8c293a44 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.314372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 927/1869] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:03.315284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.321809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 943/1870] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:03.324185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c596-71f0-436b-ae11-e8ab59539c22 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.324485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c596-71f0-436b-ae11-e8ab59539c22 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.324824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c596-71f0-436b-ae11-e8ab59539c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.335154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aef447-ae4b-4337-a274-11bb1d5ae8fc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.342392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094bf6b-9957-4cd8-a7b9-571ec31de8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.344496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094bf6b-9957-4cd8-a7b9-571ec31de8e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.345014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094bf6b-9957-4cd8-a7b9-571ec31de8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.345818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094bf6b-9957-4cd8-a7b9-571ec31de8e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.346532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3949c596-71f0-436b-ae11-e8ab59539c22 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:03.346638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094bf6b-9957-4cd8-a7b9-571ec31de8e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.347891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3949c596-71f0-436b-ae11-e8ab59539c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.348852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094bf6b-9957-4cd8-a7b9-571ec31de8e9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.361184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 944/1871] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:03.362146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949c596-71f0-436b-ae11-e8ab59539c22 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.378011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 928/1872] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/users => generated 305 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:03.378774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094bf6b-9957-4cd8-a7b9-571ec31de8e9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.388060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949c596-71f0-436b-ae11-e8ab59539c22 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.394783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094bf6b-9957-4cd8-a7b9-571ec31de8e9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.396033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c37570-6e4e-462f-8639-209bab01ce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.396616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c37570-6e4e-462f-8639-209bab01ce3b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.396988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c37570-6e4e-462f-8639-209bab01ce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.397262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c37570-6e4e-462f-8639-209bab01ce3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.399302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c37570-6e4e-462f-8639-209bab01ce3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.400085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.400808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.401160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.401478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.401571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c37570-6e4e-462f-8639-209bab01ce3b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.402267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:03.402579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.404196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.419512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 929/1873] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:03.420828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.432749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939644be-c499-446c-8463-bd73230f3292 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.438729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142ab46-155a-4697-9e11-389dca48d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.439476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142ab46-155a-4697-9e11-389dca48d916 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.439819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142ab46-155a-4697-9e11-389dca48d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.440097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 945/1874] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:03.440218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142ab46-155a-4697-9e11-389dca48d916 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.440897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c37570-6e4e-462f-8639-209bab01ce3b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.441004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142ab46-155a-4697-9e11-389dca48d916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.442613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142ab46-155a-4697-9e11-389dca48d916 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.457512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 930/1875] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:03.459049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142ab46-155a-4697-9e11-389dca48d916 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.471520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142ab46-155a-4697-9e11-389dca48d916 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.501016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:03.515980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:03.520399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c37570-6e4e-462f-8639-209bab01ce3b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.528027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.528697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.529013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.529292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.530045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.531950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.536890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8949bf-fa25-4bc7-a4fd-0195ae02afa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.537597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8949bf-fa25-4bc7-a4fd-0195ae02afa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.537882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8949bf-fa25-4bc7-a4fd-0195ae02afa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.538160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8949bf-fa25-4bc7-a4fd-0195ae02afa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.538873 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8949bf-fa25-4bc7-a4fd-0195ae02afa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.541785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8949bf-fa25-4bc7-a4fd-0195ae02afa2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.559813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 946/1876] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:03.560620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.562199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.569069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 931/1877] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:03.569069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8949bf-fa25-4bc7-a4fd-0195ae02afa2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.578938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8949bf-fa25-4bc7-a4fd-0195ae02afa2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.627553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ecf94-fcad-4edd-b124-ef2a0ddb1357 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.632343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.633207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.633488 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.633762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] PATH_INFO: `/v3/projects/af36a88886614de387192b0fa4b1870b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.634975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af36a88886614de387192b0fa4b1870b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.636890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca424f2a-3ffa-4472-b40d-362d2e17f62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.637196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.637565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424f2a-3ffa-4472-b40d-362d2e17f62d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.637852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424f2a-3ffa-4472-b40d-362d2e17f62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.638130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424f2a-3ffa-4472-b40d-362d2e17f62d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.639537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca424f2a-3ffa-4472-b40d-362d2e17f62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.641135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca424f2a-3ffa-4472-b40d-362d2e17f62d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.653160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 932/1878] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:03.654105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca424f2a-3ffa-4472-b40d-362d2e17f62d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.664353 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca424f2a-3ffa-4472-b40d-362d2e17f62d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.670264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50976aab-f609-4393-8f2d-fd1beca90ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.671020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50976aab-f609-4393-8f2d-fd1beca90ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.671400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50976aab-f609-4393-8f2d-fd1beca90ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.671690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50976aab-f609-4393-8f2d-fd1beca90ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.672408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50976aab-f609-4393-8f2d-fd1beca90ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.674324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50976aab-f609-4393-8f2d-fd1beca90ae6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.686780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af36a88886614de387192b0fa4b1870b', 'request_id': 'req-00a239ee-3e95-4dfd-b0e7-aa63626ff475'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.693374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af36a88886614de387192b0fa4b1870b', 'request_id': 'req-00a239ee-3e95-4dfd-b0e7-aa63626ff475'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.695522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 933/1879] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:03 2022] POST /identity/v3/users => generated 285 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:03.695522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50976aab-f609-4393-8f2d-fd1beca90ae6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.699910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af36a88886614de387192b0fa4b1870b', 'request_id': 'req-00a239ee-3e95-4dfd-b0e7-aa63626ff475'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.707100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50976aab-f609-4393-8f2d-fd1beca90ae6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.713422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] The token cache is being invalidate because project af36a88886614de387192b0fa4b1870b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:03.713677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.714335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.718877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 947/1880] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:03 2022] DELETE /identity/v3/projects/af36a88886614de387192b0fa4b1870b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:03.718877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.720746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.768156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.768826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.769107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.769382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.770113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.771735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.781924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 934/1881] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:03.783190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.785012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.787525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a239ee-3e95-4dfd-b0e7-aa63626ff475 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.792824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.793721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.794004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.794282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] PATH_INFO: `/v3/projects/65311989b976446ca8e8822952306bf4/users/8c2297338af04003a66d2bdb7a644f99/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.797353 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568302d4-e0b7-47e9-b32f-6f800a61395a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.801909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.802562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.802845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.803124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.803993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.805574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.809894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65311989b976446ca8e8822952306bf4, user_id=8c2297338af04003a66d2bdb7a644f99, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.812528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.816408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 935/1882] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:03.816691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:03.817680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.819584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.833454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8109602-e6de-455e-abd9-dab0ecae0250 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.836133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 948/1883] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:03 2022] PUT /identity/v3/projects/65311989b976446ca8e8822952306bf4/users/8c2297338af04003a66d2bdb7a644f99/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:03.836414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.838573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:03.855416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:03.898957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.902974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.903270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.903576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] PATH_INFO: `/v3/users/e3fba92b1eec4e0dbb743b7d82f966fe` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.904887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3fba92b1eec4e0dbb743b7d82f966fe)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.906832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.928700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd691d8c-009c-40ae-be95-9203eaf76f59 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.937586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272851c-1d2a-479d-b84f-a1e666c09b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.938275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272851c-1d2a-479d-b84f-a1e666c09b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.938703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272851c-1d2a-479d-b84f-a1e666c09b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.938978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272851c-1d2a-479d-b84f-a1e666c09b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.939839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272851c-1d2a-479d-b84f-a1e666c09b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.941406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272851c-1d2a-479d-b84f-a1e666c09b48 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:03.954861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 949/1884] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:03.959296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272851c-1d2a-479d-b84f-a1e666c09b48 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:03.970806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272851c-1d2a-479d-b84f-a1e666c09b48 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:03.980947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3fba92b1eec4e0dbb743b7d82f966fe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:03.984392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d96dd5-ed5d-40e6-b381-a4afdbf033cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:03.985206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96dd5-ed5d-40e6-b381-a4afdbf033cd admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:03.985583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96dd5-ed5d-40e6-b381-a4afdbf033cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:03.985779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96dd5-ed5d-40e6-b381-a4afdbf033cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:03.986728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d96dd5-ed5d-40e6-b381-a4afdbf033cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:03.988381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d96dd5-ed5d-40e6-b381-a4afdbf033cd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.000024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 950/1885] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:04.000289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3fba92b1eec4e0dbb743b7d82f966fe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.001390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573d0cb-354a-4070-ad02-efa152bec65e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.001682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573d0cb-354a-4070-ad02-efa152bec65e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.001964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573d0cb-354a-4070-ad02-efa152bec65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.016812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3fba92b1eec4e0dbb743b7d82f966fe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.019754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3fba92b1eec4e0dbb743b7d82f966fe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.028978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3fba92b1eec4e0dbb743b7d82f966fe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.032146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4573d0cb-354a-4070-ad02-efa152bec65e None None] MFA Rules not processed for user `8c2297338af04003a66d2bdb7a644f99`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:04.033526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4573d0cb-354a-4070-ad02-efa152bec65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.035450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 936/1886] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:03 2022] DELETE /identity/v3/users/e3fba92b1eec4e0dbb743b7d82f966fe => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:04.036232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.037719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.143832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 951/1887] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:04.144269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fc8c4-cee3-4c5e-a4ba-067076ea0d8d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.148645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573d0cb-354a-4070-ad02-efa152bec65e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.151224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4573d0cb-354a-4070-ad02-efa152bec65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.153176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.154579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.154827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.155142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] PATH_INFO: `/v3/projects/dd6efc6e9dad4ab1b9c40ab78d5a360d/users/b6a6d8ab7cc04e14b8f148412974e995/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.168595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573d0cb-354a-4070-ad02-efa152bec65e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.173771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40061e9-2106-4685-ac5b-ec4f882e8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.174021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6efc6e9dad4ab1b9c40ab78d5a360d, user_id=b6a6d8ab7cc04e14b8f148412974e995, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.174809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40061e9-2106-4685-ac5b-ec4f882e8a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.175101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40061e9-2106-4685-ac5b-ec4f882e8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.175410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40061e9-2106-4685-ac5b-ec4f882e8a8c admin admin] PATH_INFO: `/v3/projects/2fe3f1e80cad48f29980ad57654aeae8/users/c6416b8a27bb49e39c3d457bc676a613/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.177229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.194093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 937/1888] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:04 2022] PUT /identity/v3/projects/dd6efc6e9dad4ab1b9c40ab78d5a360d/users/b6a6d8ab7cc04e14b8f148412974e995/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:04.195435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.198886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.200634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40061e9-2106-4685-ac5b-ec4f882e8a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe3f1e80cad48f29980ad57654aeae8, user_id=c6416b8a27bb49e39c3d457bc676a613, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.203928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40061e9-2106-4685-ac5b-ec4f882e8a8c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.223250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 952/1889] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:04 2022] PUT /identity/v3/projects/2fe3f1e80cad48f29980ad57654aeae8/users/c6416b8a27bb49e39c3d457bc676a613/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:04.224812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21742-9564-4d58-9c6c-feb0452e5836 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.225250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21742-9564-4d58-9c6c-feb0452e5836 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.225673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21742-9564-4d58-9c6c-feb0452e5836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.236976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:04.254369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-fbb21742-9564-4d58-9c6c-feb0452e5836 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:04.255663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fbb21742-9564-4d58-9c6c-feb0452e5836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.294963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5951b-d085-4c5a-bcee-2a5ed30a3912 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.317266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:04.351013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 953/1890] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:04.352500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55dc9c-a1cc-41ac-b8d8-949cf0c12e89 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.352792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55dc9c-a1cc-41ac-b8d8-949cf0c12e89 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.353067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55dc9c-a1cc-41ac-b8d8-949cf0c12e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.384546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.385486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.385874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.386250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.387097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.387205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3f55dc9c-a1cc-41ac-b8d8-949cf0c12e89 None None] MFA Rules not processed for user `c6416b8a27bb49e39c3d457bc676a613`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:04.388421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3f55dc9c-a1cc-41ac-b8d8-949cf0c12e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.389279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.425122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 938/1891] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/projects => generated 407 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:04.429483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.431859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.476769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 954/1892] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:04.477601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55dc9c-a1cc-41ac-b8d8-949cf0c12e89 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.488998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55dc9c-a1cc-41ac-b8d8-949cf0c12e89 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.499402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d85e61-fedc-4f83-8e82-55b9e11d40a8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.503026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55c628-52c2-43c0-89bc-25d4b0a5745d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.503720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55c628-52c2-43c0-89bc-25d4b0a5745d service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.503997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55c628-52c2-43c0-89bc-25d4b0a5745d service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.504273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55c628-52c2-43c0-89bc-25d4b0a5745d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.546207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.547100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.547238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.547460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.548148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.550082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.566766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55c628-52c2-43c0-89bc-25d4b0a5745d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.568214 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55c628-52c2-43c0-89bc-25d4b0a5745d service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.571490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 955/1893] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/users => generated 310 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:04.572513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.574669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.581618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 939/1894] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:04.582423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55c628-52c2-43c0-89bc-25d4b0a5745d service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.588389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:04.589759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753945f2-0aeb-447c-965e-218792b04885 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.592705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55c628-52c2-43c0-89bc-25d4b0a5745d service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.597193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.598680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.598969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.599135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e6ed3-4829-4102-8e05-38fcc0d900a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.599686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] PATH_INFO: `/v3/projects/dfeeb42a6826496e8b0c08bd499fe41d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.599808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e6ed3-4829-4102-8e05-38fcc0d900a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.600203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e6ed3-4829-4102-8e05-38fcc0d900a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.600593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e6ed3-4829-4102-8e05-38fcc0d900a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.601139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfeeb42a6826496e8b0c08bd499fe41d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.601449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e6ed3-4829-4102-8e05-38fcc0d900a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.602880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.603463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e6ed3-4829-4102-8e05-38fcc0d900a7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.636927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 940/1895] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:04.640306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e6ed3-4829-4102-8e05-38fcc0d900a7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.664016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfeeb42a6826496e8b0c08bd499fe41d', 'request_id': 'req-995c49fc-97d9-4500-87c9-90cd1decad03'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.672921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfeeb42a6826496e8b0c08bd499fe41d', 'request_id': 'req-995c49fc-97d9-4500-87c9-90cd1decad03'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.678800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:04.679580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfeeb42a6826496e8b0c08bd499fe41d', 'request_id': 'req-995c49fc-97d9-4500-87c9-90cd1decad03'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.697573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] The token cache is being invalidate because project dfeeb42a6826496e8b0c08bd499fe41d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:04.697893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.698519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:04.700697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 956/1896] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:04 2022] DELETE /identity/v3/projects/dfeeb42a6826496e8b0c08bd499fe41d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:04.701259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.702637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.719438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e6ed3-4829-4102-8e05-38fcc0d900a7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.797411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c49fc-97d9-4500-87c9-90cd1decad03 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.802202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.802886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.803543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.803543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.804157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.804419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17353b-4e36-48d4-ac8b-6f0273844a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.805102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17353b-4e36-48d4-ac8b-6f0273844a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.805705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17353b-4e36-48d4-ac8b-6f0273844a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.805948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17353b-4e36-48d4-ac8b-6f0273844a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.806109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.807219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17353b-4e36-48d4-ac8b-6f0273844a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.809614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17353b-4e36-48d4-ac8b-6f0273844a16 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.822975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 941/1897] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:04.824354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce05e00-5715-4113-9089-35f72d3602ca None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.824641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce05e00-5715-4113-9089-35f72d3602ca None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.825119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce05e00-5715-4113-9089-35f72d3602ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.835028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 957/1898] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:04.836453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.840498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.845712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8ce05e00-5715-4113-9089-35f72d3602ca None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:04.846914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8ce05e00-5715-4113-9089-35f72d3602ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:04.856231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae91b9-68cc-4ad6-933b-7bfedaae0222 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.856695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 942/1899] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:04.857743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce05e00-5715-4113-9089-35f72d3602ca None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.868198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce05e00-5715-4113-9089-35f72d3602ca None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:04.869161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f679a6-7691-4bbd-9039-e781f4109dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.871490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f679a6-7691-4bbd-9039-e781f4109dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.871726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f679a6-7691-4bbd-9039-e781f4109dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.872016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f679a6-7691-4bbd-9039-e781f4109dd9 admin admin] PATH_INFO: `/v3/projects/36de2dcc992e4733885ccb064d22d81a/users/d733141c7a834049bb5e769cc4b5f934/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.874647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad55e3-83cf-4f39-9157-af1953840f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:04.875331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad55e3-83cf-4f39-9157-af1953840f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.875647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad55e3-83cf-4f39-9157-af1953840f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.875928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad55e3-83cf-4f39-9157-af1953840f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.876666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad55e3-83cf-4f39-9157-af1953840f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.878273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad55e3-83cf-4f39-9157-af1953840f4d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.891327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f679a6-7691-4bbd-9039-e781f4109dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36de2dcc992e4733885ccb064d22d81a, user_id=d733141c7a834049bb5e769cc4b5f934, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:04.894115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 943/1900] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:04.894229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f679a6-7691-4bbd-9039-e781f4109dd9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:04.895732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4266f92-bbc7-4818-b82f-78afbabdee61 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:04.896179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4266f92-bbc7-4818-b82f-78afbabdee61 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:04.897026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4266f92-bbc7-4818-b82f-78afbabdee61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:04.911536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 958/1901] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:04 2022] PUT /identity/v3/projects/36de2dcc992e4733885ccb064d22d81a/users/d733141c7a834049bb5e769cc4b5f934/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:04.912497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f679a6-7691-4bbd-9039-e781f4109dd9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:04.921621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d4266f92-bbc7-4818-b82f-78afbabdee61 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:04.922784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d4266f92-bbc7-4818-b82f-78afbabdee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:05.018698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f679a6-7691-4bbd-9039-e781f4109dd9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.028607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7042e9d-6982-4d94-8207-7fa39db1d572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.029561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7042e9d-6982-4d94-8207-7fa39db1d572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.029837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7042e9d-6982-4d94-8207-7fa39db1d572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.030124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7042e9d-6982-4d94-8207-7fa39db1d572 admin admin] PATH_INFO: `/v3/projects/8c171927891346b688d93e772428e839/users/4a02baae8582483fab2f8ff9c85e5190/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.040991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 944/1902] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:05.043117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4266f92-bbc7-4818-b82f-78afbabdee61 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.055626 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4266f92-bbc7-4818-b82f-78afbabdee61 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.057805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7042e9d-6982-4d94-8207-7fa39db1d572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c171927891346b688d93e772428e839, user_id=4a02baae8582483fab2f8ff9c85e5190, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.061095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58a6f-43f5-4265-914f-8a2b0bca0bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.061812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58a6f-43f5-4265-914f-8a2b0bca0bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.061985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7042e9d-6982-4d94-8207-7fa39db1d572 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.062250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58a6f-43f5-4265-914f-8a2b0bca0bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.062673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58a6f-43f5-4265-914f-8a2b0bca0bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.063592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac58a6f-43f5-4265-914f-8a2b0bca0bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.065186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac58a6f-43f5-4265-914f-8a2b0bca0bf3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.068301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:05.080306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 959/1903] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:04 2022] PUT /identity/v3/projects/8c171927891346b688d93e772428e839/users/4a02baae8582483fab2f8ff9c85e5190/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:05.081274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7042e9d-6982-4d94-8207-7fa39db1d572 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.083614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 945/1904] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:05.086306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58a6f-43f5-4265-914f-8a2b0bca0bf3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.097456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58a6f-43f5-4265-914f-8a2b0bca0bf3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.136812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:05.169693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf6e54-1fe8-4dde-9b4f-af7f599de98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.170657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf6e54-1fe8-4dde-9b4f-af7f599de98e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.170945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf6e54-1fe8-4dde-9b4f-af7f599de98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.171226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf6e54-1fe8-4dde-9b4f-af7f599de98e admin admin] PATH_INFO: `/v3/projects/36de2dcc992e4733885ccb064d22d81a/users/d733141c7a834049bb5e769cc4b5f934/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.174087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf6e54-1fe8-4dde-9b4f-af7f599de98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=36de2dcc992e4733885ccb064d22d81a, user_id=d733141c7a834049bb5e769cc4b5f934, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.177112 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf6e54-1fe8-4dde-9b4f-af7f599de98e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.180258 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7042e9d-6982-4d94-8207-7fa39db1d572 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.186959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.187674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.188103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.188553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.189594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:05.189594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.191220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.200163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 946/1905] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:05 2022] PUT /identity/v3/projects/36de2dcc992e4733885ccb064d22d81a/users/d733141c7a834049bb5e769cc4b5f934/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:05.202414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf6e54-1fe8-4dde-9b4f-af7f599de98e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.207433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 960/1906] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:05.213917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.227587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e1045-7926-41c4-b2b9-00382cdec7b3 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.313560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf6e54-1fe8-4dde-9b4f-af7f599de98e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.322797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280521e-2033-4e46-b651-0622509e2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.323452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90450e-8bd2-4fd7-8382-d45de04ac7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.323652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280521e-2033-4e46-b651-0622509e2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.324067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280521e-2033-4e46-b651-0622509e2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.324435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280521e-2033-4e46-b651-0622509e2bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.325466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280521e-2033-4e46-b651-0622509e2bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.325765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90450e-8bd2-4fd7-8382-d45de04ac7bf admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.326229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90450e-8bd2-4fd7-8382-d45de04ac7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.326903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90450e-8bd2-4fd7-8382-d45de04ac7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.328004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90450e-8bd2-4fd7-8382-d45de04ac7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.328393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280521e-2033-4e46-b651-0622509e2bf0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.337277 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90450e-8bd2-4fd7-8382-d45de04ac7bf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.344623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 947/1907] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:05.346139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d933230-19e4-4b72-bce7-cd222ca30d7f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.346260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d933230-19e4-4b72-bce7-cd222ca30d7f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.346546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d933230-19e4-4b72-bce7-cd222ca30d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.394010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8d933230-19e4-4b72-bce7-cd222ca30d7f None None] MFA Rules not processed for user `d733141c7a834049bb5e769cc4b5f934`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:05.395377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8d933230-19e4-4b72-bce7-cd222ca30d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:05.397027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 961/1908] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:05 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:05.398075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90450e-8bd2-4fd7-8382-d45de04ac7bf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.485162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90450e-8bd2-4fd7-8382-d45de04ac7bf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.491255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d17c9-87b1-4026-b797-8d94f83604ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.492180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d17c9-87b1-4026-b797-8d94f83604ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.492497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d17c9-87b1-4026-b797-8d94f83604ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.492776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d17c9-87b1-4026-b797-8d94f83604ea admin admin] PATH_INFO: `/v3/projects/8c171927891346b688d93e772428e839/users/4a02baae8582483fab2f8ff9c85e5190/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.495710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d17c9-87b1-4026-b797-8d94f83604ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c171927891346b688d93e772428e839, user_id=4a02baae8582483fab2f8ff9c85e5190, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.499715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 948/1909] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:05 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:05.501597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6099a-7884-4942-9688-443623c5eec6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.502066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6099a-7884-4942-9688-443623c5eec6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.502930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6099a-7884-4942-9688-443623c5eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.503797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d17c9-87b1-4026-b797-8d94f83604ea admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.531765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 962/1910] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:05 2022] PUT /identity/v3/projects/8c171927891346b688d93e772428e839/users/4a02baae8582483fab2f8ff9c85e5190/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:05.533131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d17c9-87b1-4026-b797-8d94f83604ea admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.554597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:05.560111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e2d6099a-7884-4942-9688-443623c5eec6 None None] MFA Rules not processed for user `b6a6d8ab7cc04e14b8f148412974e995`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:05.562598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e2d6099a-7884-4942-9688-443623c5eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:05.606820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:05.624696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d17c9-87b1-4026-b797-8d94f83604ea admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.629369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62238d7c-3ec7-4d68-b482-65e260dbd15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.630036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62238d7c-3ec7-4d68-b482-65e260dbd15a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.630317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62238d7c-3ec7-4d68-b482-65e260dbd15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.630593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62238d7c-3ec7-4d68-b482-65e260dbd15a admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.631281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62238d7c-3ec7-4d68-b482-65e260dbd15a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.633268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62238d7c-3ec7-4d68-b482-65e260dbd15a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.640030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 949/1911] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:05.641223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6099a-7884-4942-9688-443623c5eec6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.642590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e2d6099a-7884-4942-9688-443623c5eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:05.657140 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 963/1912] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:05 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:05.658354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537738ec-1f82-447c-b309-2a324d8a33ae None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.658702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537738ec-1f82-447c-b309-2a324d8a33ae None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.659037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537738ec-1f82-447c-b309-2a324d8a33ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.688475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-537738ec-1f82-447c-b309-2a324d8a33ae None None] MFA Rules not processed for user `4a02baae8582483fab2f8ff9c85e5190`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:05.689688 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-537738ec-1f82-447c-b309-2a324d8a33ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:05.711180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6099a-7884-4942-9688-443623c5eec6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.715872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461da12-395a-46fd-a7d2-05ad8ec6160c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.716543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461da12-395a-46fd-a7d2-05ad8ec6160c service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.716832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461da12-395a-46fd-a7d2-05ad8ec6160c service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.717115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461da12-395a-46fd-a7d2-05ad8ec6160c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.761657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 964/1913] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:05 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:05.762583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537738ec-1f82-447c-b309-2a324d8a33ae None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.763915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-537738ec-1f82-447c-b309-2a324d8a33ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:05.776157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537738ec-1f82-447c-b309-2a324d8a33ae None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.780118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4d4e7-5085-4cbe-81d7-ac7499380b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.780747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4d4e7-5085-4cbe-81d7-ac7499380b11 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.781044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4d4e7-5085-4cbe-81d7-ac7499380b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.781319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4d4e7-5085-4cbe-81d7-ac7499380b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.787278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461da12-395a-46fd-a7d2-05ad8ec6160c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.788751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461da12-395a-46fd-a7d2-05ad8ec6160c service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.790055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4d4e7-5085-4cbe-81d7-ac7499380b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.791781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4d4e7-5085-4cbe-81d7-ac7499380b11 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.798376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 950/1914] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:05 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:05.799165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461da12-395a-46fd-a7d2-05ad8ec6160c service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.800741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 965/1915] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:05.814278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461da12-395a-46fd-a7d2-05ad8ec6160c service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.820323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055659c5-4bea-41ee-b1d7-65e6de098041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.820998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055659c5-4bea-41ee-b1d7-65e6de098041 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.821285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055659c5-4bea-41ee-b1d7-65e6de098041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.821566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055659c5-4bea-41ee-b1d7-65e6de098041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.822848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055659c5-4bea-41ee-b1d7-65e6de098041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.824432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055659c5-4bea-41ee-b1d7-65e6de098041 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.839424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 951/1916] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:05.840538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4d4e7-5085-4cbe-81d7-ac7499380b11 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.852965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4d4e7-5085-4cbe-81d7-ac7499380b11 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.854552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055659c5-4bea-41ee-b1d7-65e6de098041 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.857926 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69751453-4d80-492b-8e3d-787b773dc035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.858602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69751453-4d80-492b-8e3d-787b773dc035 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.858886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69751453-4d80-492b-8e3d-787b773dc035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.859163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69751453-4d80-492b-8e3d-787b773dc035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.859950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69751453-4d80-492b-8e3d-787b773dc035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.861665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69751453-4d80-492b-8e3d-787b773dc035 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.865555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055659c5-4bea-41ee-b1d7-65e6de098041 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.871178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d36a95-80c4-49de-b816-c9c07ed88107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.871955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36a95-80c4-49de-b816-c9c07ed88107 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.872316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36a95-80c4-49de-b816-c9c07ed88107 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.872601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36a95-80c4-49de-b816-c9c07ed88107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.876891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 966/1917] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:05.877908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69751453-4d80-492b-8e3d-787b773dc035 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:05.885344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d36a95-80c4-49de-b816-c9c07ed88107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.886828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d36a95-80c4-49de-b816-c9c07ed88107 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.888360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69751453-4d80-492b-8e3d-787b773dc035 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:05.893153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b678-c356-4261-a72a-7918643e8cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:05.894031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b678-c356-4261-a72a-7918643e8cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.894315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b678-c356-4261-a72a-7918643e8cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.894589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b678-c356-4261-a72a-7918643e8cfa admin admin] PATH_INFO: `/v3/projects/cd1abbfb21fe4492b33c134f3400a99a/users/75af86ed2d4f4c2a8ca3cf44c153f457/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.897716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 952/1918] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:05 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:05.912309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2b678-c356-4261-a72a-7918643e8cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1abbfb21fe4492b33c134f3400a99a, user_id=75af86ed2d4f4c2a8ca3cf44c153f457, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:05.914983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2b678-c356-4261-a72a-7918643e8cfa admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:05.933381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 967/1919] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:05 2022] PUT /identity/v3/projects/cd1abbfb21fe4492b33c134f3400a99a/users/75af86ed2d4f4c2a8ca3cf44c153f457/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:05.945144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150abe1-7390-4113-be2d-3f279d1e9808 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:05.945285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150abe1-7390-4113-be2d-3f279d1e9808 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:05.945568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150abe1-7390-4113-be2d-3f279d1e9808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:05.984534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3150abe1-7390-4113-be2d-3f279d1e9808 None None] MFA Rules not processed for user `75af86ed2d4f4c2a8ca3cf44c153f457`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:05.986341 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3150abe1-7390-4113-be2d-3f279d1e9808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:06.009103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:06.069785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 953/1920] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:06.074884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b678-c356-4261-a72a-7918643e8cfa admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:06.104270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150abe1-7390-4113-be2d-3f279d1e9808 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:06.161986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b678-c356-4261-a72a-7918643e8cfa admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:06.165839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150abe1-7390-4113-be2d-3f279d1e9808 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:06.168167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487e22-1ac4-408a-8d22-b3ed6ea632c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:06.168846 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487e22-1ac4-408a-8d22-b3ed6ea632c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.169131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487e22-1ac4-408a-8d22-b3ed6ea632c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.169408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487e22-1ac4-408a-8d22-b3ed6ea632c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.170239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c487e22-1ac4-408a-8d22-b3ed6ea632c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:06.170327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:06.171089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.171570 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.171957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.172252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c487e22-1ac4-408a-8d22-b3ed6ea632c4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:06.173991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:06.205303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 968/1921] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:06 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:06.206445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487e22-1ac4-408a-8d22-b3ed6ea632c4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:06.212193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:06.237978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:06.239484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:06.249583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 954/1922] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:06.250382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:06.264146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985e075-3d9d-4796-aae0-758b70a27ca1 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:06.271055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487e22-1ac4-408a-8d22-b3ed6ea632c4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:06.275136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:06.276008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.276284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.276560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.278826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:06.334768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95beb52-8f00-412e-9f2d-3ce337d34479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:06.335574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95beb52-8f00-412e-9f2d-3ce337d34479 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.335933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95beb52-8f00-412e-9f2d-3ce337d34479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.336280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95beb52-8f00-412e-9f2d-3ce337d34479 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.337202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95beb52-8f00-412e-9f2d-3ce337d34479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:06.339676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95beb52-8f00-412e-9f2d-3ce337d34479 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:06.373610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 955/1923] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:06 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:06.380604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95beb52-8f00-412e-9f2d-3ce337d34479 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:06.396044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:06.409259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95beb52-8f00-412e-9f2d-3ce337d34479 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:06.416291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:06.420503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:06.435643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd903d-3a94-47dd-8db8-e91e8547871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:06.436769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd903d-3a94-47dd-8db8-e91e8547871c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.437198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd903d-3a94-47dd-8db8-e91e8547871c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.437614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd903d-3a94-47dd-8db8-e91e8547871c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.438494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd903d-3a94-47dd-8db8-e91e8547871c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:06.441323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 969/1924] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:06.443619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd903d-3a94-47dd-8db8-e91e8547871c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:06.472498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 956/1925] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:06.476412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:06.492667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd8245-c7a2-41d8-80ca-e2b2cbd48bc0 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:06.503880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d7f3-e78d-43b4-a727-e0f4ac01fe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:06.504925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d7f3-e78d-43b4-a727-e0f4ac01fe62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.505205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d7f3-e78d-43b4-a727-e0f4ac01fe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.505749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d7f3-e78d-43b4-a727-e0f4ac01fe62 admin admin] PATH_INFO: `/v3/projects/357e4787bcca44018b516f28c7f3124e/users/669bfea4fa28423e874fd7861cbf9483/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.548874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229d7f3-e78d-43b4-a727-e0f4ac01fe62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=357e4787bcca44018b516f28c7f3124e, user_id=669bfea4fa28423e874fd7861cbf9483, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:06.552995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229d7f3-e78d-43b4-a727-e0f4ac01fe62 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:06.601293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 970/1926] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:06 2022] PUT /identity/v3/projects/357e4787bcca44018b516f28c7f3124e/users/669bfea4fa28423e874fd7861cbf9483/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:06.606128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd903d-3a94-47dd-8db8-e91e8547871c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:06.624007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd903d-3a94-47dd-8db8-e91e8547871c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:06.766659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768900c-25a7-49b0-a9a1-dc758648c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:06.767119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768900c-25a7-49b0-a9a1-dc758648c755 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.769636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768900c-25a7-49b0-a9a1-dc758648c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.769930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768900c-25a7-49b0-a9a1-dc758648c755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.772440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768900c-25a7-49b0-a9a1-dc758648c755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:06.777912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768900c-25a7-49b0-a9a1-dc758648c755 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:06.804742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 957/1927] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:06.805898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d7f3-e78d-43b4-a727-e0f4ac01fe62 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:06.818414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d7f3-e78d-43b4-a727-e0f4ac01fe62 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:06.823857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f416e6-e9c1-43cd-b260-4968ace55a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:06.824817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f416e6-e9c1-43cd-b260-4968ace55a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.825105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f416e6-e9c1-43cd-b260-4968ace55a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.825448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f416e6-e9c1-43cd-b260-4968ace55a66 admin admin] PATH_INFO: `/v3/projects/357e4787bcca44018b516f28c7f3124e/users/669bfea4fa28423e874fd7861cbf9483/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.828453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f416e6-e9c1-43cd-b260-4968ace55a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=357e4787bcca44018b516f28c7f3124e, user_id=669bfea4fa28423e874fd7861cbf9483, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:06.831625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f416e6-e9c1-43cd-b260-4968ace55a66 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:06.846642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 971/1928] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:06 2022] PUT /identity/v3/projects/357e4787bcca44018b516f28c7f3124e/users/669bfea4fa28423e874fd7861cbf9483/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:06.859511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434512b-9e24-4ec3-982f-8efee2854835 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:06.859976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434512b-9e24-4ec3-982f-8efee2854835 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:06.860402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434512b-9e24-4ec3-982f-8efee2854835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:06.912952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b434512b-9e24-4ec3-982f-8efee2854835 None None] MFA Rules not processed for user `669bfea4fa28423e874fd7861cbf9483`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:06.914216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b434512b-9e24-4ec3-982f-8efee2854835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:06.998921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 958/1929] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:07.006953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f416e6-e9c1-43cd-b260-4968ace55a66 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:07.066207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f416e6-e9c1-43cd-b260-4968ace55a66 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:07.069983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b8896-1a12-4bd0-b607-8681e14658fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:07.070630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b8896-1a12-4bd0-b607-8681e14658fb service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:07.070915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b8896-1a12-4bd0-b607-8681e14658fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:07.071195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b8896-1a12-4bd0-b607-8681e14658fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:07.080916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b8896-1a12-4bd0-b607-8681e14658fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:07.082308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b8896-1a12-4bd0-b607-8681e14658fb service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:07.091702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 972/1930] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:07.155037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434512b-9e24-4ec3-982f-8efee2854835 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:07.216055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434512b-9e24-4ec3-982f-8efee2854835 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:07.220644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:07.221522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:07.221802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:07.222077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] PATH_INFO: `/v3/users/75af86ed2d4f4c2a8ca3cf44c153f457` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:07.223279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75af86ed2d4f4c2a8ca3cf44c153f457)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:07.225235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:07.238503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:07.271129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75af86ed2d4f4c2a8ca3cf44c153f457'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:07.278574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75af86ed2d4f4c2a8ca3cf44c153f457'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:07.298290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75af86ed2d4f4c2a8ca3cf44c153f457'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:07.299176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75af86ed2d4f4c2a8ca3cf44c153f457'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:07.308012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75af86ed2d4f4c2a8ca3cf44c153f457'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:07.314056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 959/1931] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:07 2022] DELETE /identity/v3/users/75af86ed2d4f4c2a8ca3cf44c153f457 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:07.318557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8f32-8858-4d2d-9f28-5d6c70ce6e7e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:07.318844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8f32-8858-4d2d-9f28-5d6c70ce6e7e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:07.319120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8f32-8858-4d2d-9f28-5d6c70ce6e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:07.339188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ea1b8f32-8858-4d2d-9f28-5d6c70ce6e7e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:07.340395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ea1b8f32-8858-4d2d-9f28-5d6c70ce6e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:07.374167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:07.468832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 973/1932] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:07.490629 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:07.492108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:07.569632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ee127-cedd-4cc6-a47f-0ff46c52bd05 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:07.573529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:07.574188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:07.574499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:07.574768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:07.584916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:07.586359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:07.599016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 960/1933] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:07.893571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31949a7-9ca2-4529-a2ad-c6d1efbce73d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:07.893571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31949a7-9ca2-4529-a2ad-c6d1efbce73d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:07.893571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31949a7-9ca2-4529-a2ad-c6d1efbce73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:07.908640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c31949a7-9ca2-4529-a2ad-c6d1efbce73d None None] MFA Rules not processed for user `b6a6d8ab7cc04e14b8f148412974e995`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:07.909881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c31949a7-9ca2-4529-a2ad-c6d1efbce73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:08.014302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 974/1934] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:08.837146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:08.840391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:08.855832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2bed6-5a08-4375-964f-40f782154dee service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:08.861597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:08.862518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:08.862860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:08.863138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] PATH_INFO: `/v3/projects/cd1abbfb21fe4492b33c134f3400a99a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:08.864520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd1abbfb21fe4492b33c134f3400a99a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:08.866635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:08.957691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1abbfb21fe4492b33c134f3400a99a', 'request_id': 'req-7daaf097-871f-4ed6-ad62-b2c2955f41c6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:08.976697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1abbfb21fe4492b33c134f3400a99a', 'request_id': 'req-7daaf097-871f-4ed6-ad62-b2c2955f41c6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:08.986204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1abbfb21fe4492b33c134f3400a99a', 'request_id': 'req-7daaf097-871f-4ed6-ad62-b2c2955f41c6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.001643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] The token cache is being invalidate because project cd1abbfb21fe4492b33c134f3400a99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:09.002315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.002767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.004760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 961/1935] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:08 2022] DELETE /identity/v3/projects/cd1abbfb21fe4492b33c134f3400a99a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:09.008550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31949a7-9ca2-4529-a2ad-c6d1efbce73d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:09.021734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31949a7-9ca2-4529-a2ad-c6d1efbce73d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:09.083787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:09.084659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:09.084934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:09.085225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] PATH_INFO: `/v3/users/669bfea4fa28423e874fd7861cbf9483` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:09.086418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=669bfea4fa28423e874fd7861cbf9483)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:09.088425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:09.136462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '669bfea4fa28423e874fd7861cbf9483'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.137149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '669bfea4fa28423e874fd7861cbf9483'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.144023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '669bfea4fa28423e874fd7861cbf9483'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.155070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:09.157507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '669bfea4fa28423e874fd7861cbf9483'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.158483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f3e79b0-4f8e-4712-bdcc-51bd9d58e636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '669bfea4fa28423e874fd7861cbf9483'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.179431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 975/1936] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:09 2022] DELETE /identity/v3/users/669bfea4fa28423e874fd7861cbf9483 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:09.589980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:09.591469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:09.612047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf097-871f-4ed6-ad62-b2c2955f41c6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:09.628978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:09.678967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:09.679794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:09.680077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:09.680409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] PATH_INFO: `/v3/projects/357e4787bcca44018b516f28c7f3124e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:09.681654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=357e4787bcca44018b516f28c7f3124e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:09.683602 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:09.738905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '357e4787bcca44018b516f28c7f3124e', 'request_id': 'req-db131ed6-ace5-401d-8d1a-05c94227d1e1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.746577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '357e4787bcca44018b516f28c7f3124e', 'request_id': 'req-db131ed6-ace5-401d-8d1a-05c94227d1e1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.753058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '357e4787bcca44018b516f28c7f3124e', 'request_id': 'req-db131ed6-ace5-401d-8d1a-05c94227d1e1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.766322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] The token cache is being invalidate because project 357e4787bcca44018b516f28c7f3124e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:09.766623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.767273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db131ed6-ace5-401d-8d1a-05c94227d1e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:09.768894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 962/1937] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:09 2022] DELETE /identity/v3/projects/357e4787bcca44018b516f28c7f3124e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:09.778222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736b02e-0332-4727-8870-8e5c00bfb966 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:09.778533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736b02e-0332-4727-8870-8e5c00bfb966 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:09.778797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736b02e-0332-4727-8870-8e5c00bfb966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:09.796392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7736b02e-0332-4727-8870-8e5c00bfb966 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:09.797615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7736b02e-0332-4727-8870-8e5c00bfb966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:09.856681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 976/1938] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:09.864495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c8635-55ab-48fc-a597-4356782fe385 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:09.864984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c8635-55ab-48fc-a597-4356782fe385 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:09.865077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c8635-55ab-48fc-a597-4356782fe385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:09.884034 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cb9c8635-55ab-48fc-a597-4356782fe385 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:09.885953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cb9c8635-55ab-48fc-a597-4356782fe385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:09.893872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 963/1939] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:09.897917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736b02e-0332-4727-8870-8e5c00bfb966 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:09.907644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736b02e-0332-4727-8870-8e5c00bfb966 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:09.912148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:09.912793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:09.913070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:09.913344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:09.914066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:09.914368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:09.916038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:09.927287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 977/1940] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:09.931318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c8635-55ab-48fc-a597-4356782fe385 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:09.940950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c8635-55ab-48fc-a597-4356782fe385 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:09.945562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591013ce-179d-4092-a53e-acddde6c4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:09.946209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591013ce-179d-4092-a53e-acddde6c4310 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:09.946496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591013ce-179d-4092-a53e-acddde6c4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:09.946774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591013ce-179d-4092-a53e-acddde6c4310 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:09.948091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591013ce-179d-4092-a53e-acddde6c4310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:09.949996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591013ce-179d-4092-a53e-acddde6c4310 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:09.978067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 964/1941] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:09 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:09.981936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.001633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa27c0-0cf8-427c-9130-8889f92901eb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.061913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3f8e3-76b1-4f28-85c2-68622b871573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.062594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f8e3-76b1-4f28-85c2-68622b871573 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.062879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f8e3-76b1-4f28-85c2-68622b871573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.063154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3f8e3-76b1-4f28-85c2-68622b871573 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.064210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3f8e3-76b1-4f28-85c2-68622b871573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.066177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3f8e3-76b1-4f28-85c2-68622b871573 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.087307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 978/1942] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:09 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:10.091414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591013ce-179d-4092-a53e-acddde6c4310 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.106006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591013ce-179d-4092-a53e-acddde6c4310 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.106270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8c556-a2c7-4b8f-ae5a-cd2c21dab0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.106948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8c556-a2c7-4b8f-ae5a-cd2c21dab0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.107235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8c556-a2c7-4b8f-ae5a-cd2c21dab0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.107631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8c556-a2c7-4b8f-ae5a-cd2c21dab0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.108274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8c556-a2c7-4b8f-ae5a-cd2c21dab0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.109917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8c556-a2c7-4b8f-ae5a-cd2c21dab0e6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.124417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 965/1943] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:10.125828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3f8e3-76b1-4f28-85c2-68622b871573 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.148703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3f8e3-76b1-4f28-85c2-68622b871573 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.153701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd304cc-63fd-45ce-91e9-351fd86951fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.154371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd304cc-63fd-45ce-91e9-351fd86951fe admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.154651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd304cc-63fd-45ce-91e9-351fd86951fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.154923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd304cc-63fd-45ce-91e9-351fd86951fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.155714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd304cc-63fd-45ce-91e9-351fd86951fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.157296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd304cc-63fd-45ce-91e9-351fd86951fe admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.171539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 979/1944] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:10.172665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8c556-a2c7-4b8f-ae5a-cd2c21dab0e6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.183294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8c556-a2c7-4b8f-ae5a-cd2c21dab0e6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.188124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5fe2-f12a-4cda-ad2c-fe19d2b886aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.189067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5fe2-f12a-4cda-ad2c-fe19d2b886aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.189352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5fe2-f12a-4cda-ad2c-fe19d2b886aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.189630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5fe2-f12a-4cda-ad2c-fe19d2b886aa admin admin] PATH_INFO: `/v3/projects/8ec9d15a03184e2c908f97e69a0736f6/users/4c559a715c7a4d3e8e28817cdc74ec9b/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.205984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd5fe2-f12a-4cda-ad2c-fe19d2b886aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec9d15a03184e2c908f97e69a0736f6, user_id=4c559a715c7a4d3e8e28817cdc74ec9b, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.208684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd5fe2-f12a-4cda-ad2c-fe19d2b886aa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.222395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 966/1945] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:10 2022] PUT /identity/v3/projects/8ec9d15a03184e2c908f97e69a0736f6/users/4c559a715c7a4d3e8e28817cdc74ec9b/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:10.234642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f2545-80ba-4196-9444-083cb37516f3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.234981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f2545-80ba-4196-9444-083cb37516f3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.235286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f2545-80ba-4196-9444-083cb37516f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.268378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b17f2545-80ba-4196-9444-083cb37516f3 None None] MFA Rules not processed for user `4c559a715c7a4d3e8e28817cdc74ec9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:10.269642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b17f2545-80ba-4196-9444-083cb37516f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:10.316481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:10.357000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 980/1946] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:10.360830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5fe2-f12a-4cda-ad2c-fe19d2b886aa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.418370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:10.439725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5fe2-f12a-4cda-ad2c-fe19d2b886aa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.444365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.445029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.445311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.445586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.446279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.448253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.481768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 967/1947] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:10 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:10.483909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f2545-80ba-4196-9444-083cb37516f3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.497352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f2545-80ba-4196-9444-083cb37516f3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.593266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589063d-51de-4b4b-9f22-5c435b5b2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.594125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589063d-51de-4b4b-9f22-5c435b5b2e7d admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.594495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589063d-51de-4b4b-9f22-5c435b5b2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.594857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589063d-51de-4b4b-9f22-5c435b5b2e7d admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.595676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589063d-51de-4b4b-9f22-5c435b5b2e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.604944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589063d-51de-4b4b-9f22-5c435b5b2e7d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.622128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.624179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:10.643791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 981/1948] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:10 2022] POST /identity/v3/users => generated 295 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:10.643941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b181d-5229-42bb-84bc-8bb0f0809d7f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.644996 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589063d-51de-4b4b-9f22-5c435b5b2e7d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.658906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.659739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589063d-51de-4b4b-9f22-5c435b5b2e7d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.659882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.660105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.660388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] PATH_INFO: `/v3/users/c6416b8a27bb49e39c3d457bc676a613` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.661634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6416b8a27bb49e39c3d457bc676a613)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.663644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.667389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c270b-52c9-44aa-842f-dcd93c3fb4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.668337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c270b-52c9-44aa-842f-dcd93c3fb4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.668981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c270b-52c9-44aa-842f-dcd93c3fb4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.669350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c270b-52c9-44aa-842f-dcd93c3fb4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.670197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c270b-52c9-44aa-842f-dcd93c3fb4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.672058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c270b-52c9-44aa-842f-dcd93c3fb4a8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.683387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 982/1949] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:10.687872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c270b-52c9-44aa-842f-dcd93c3fb4a8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.698028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c270b-52c9-44aa-842f-dcd93c3fb4a8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.703569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b48b9-a56c-4784-ba5b-36b52b9931b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.704626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b48b9-a56c-4784-ba5b-36b52b9931b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.704879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b48b9-a56c-4784-ba5b-36b52b9931b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.705172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b48b9-a56c-4784-ba5b-36b52b9931b1 admin admin] PATH_INFO: `/v3/projects/a964bc2e302b4652ade830aa5d87c2d9/users/957c16574b464b49980f686e6cc2a002/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.724964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6416b8a27bb49e39c3d457bc676a613'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:10.732526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b48b9-a56c-4784-ba5b-36b52b9931b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a964bc2e302b4652ade830aa5d87c2d9, user_id=957c16574b464b49980f686e6cc2a002, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.735255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b48b9-a56c-4784-ba5b-36b52b9931b1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.742398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6416b8a27bb49e39c3d457bc676a613'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:10.749478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 983/1950] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:10 2022] PUT /identity/v3/projects/a964bc2e302b4652ade830aa5d87c2d9/users/957c16574b464b49980f686e6cc2a002/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:10.753218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b48b9-a56c-4784-ba5b-36b52b9931b1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.757981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6416b8a27bb49e39c3d457bc676a613'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:10.758678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6416b8a27bb49e39c3d457bc676a613'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:10.766321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d812842-5bce-412f-89e2-6b017906827f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6416b8a27bb49e39c3d457bc676a613'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:10.772281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 968/1951] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:10 2022] DELETE /identity/v3/users/c6416b8a27bb49e39c3d457bc676a613 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:10.777442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fed24e-15a5-4358-a59c-9fa929296f76 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.777814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fed24e-15a5-4358-a59c-9fa929296f76 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.778019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fed24e-15a5-4358-a59c-9fa929296f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.797208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a7fed24e-15a5-4358-a59c-9fa929296f76 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:10.798505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a7fed24e-15a5-4358-a59c-9fa929296f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:10.824390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b48b9-a56c-4784-ba5b-36b52b9931b1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.866590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 969/1952] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:10.875207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:10.876814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fed24e-15a5-4358-a59c-9fa929296f76 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.878562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a7fed24e-15a5-4358-a59c-9fa929296f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:10.894538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.895372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.895696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.895981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.896855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.898629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.911679 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 984/1953] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:10.915804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:10.918843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:10.932886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62d565-96c0-4d4e-98ac-c72327ffc251 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.935889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:10.938408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9248fb0-1d21-4d44-abf5-76ea65308b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.939308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9248fb0-1d21-4d44-abf5-76ea65308b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.939635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9248fb0-1d21-4d44-abf5-76ea65308b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.939913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9248fb0-1d21-4d44-abf5-76ea65308b7e admin admin] PATH_INFO: `/v3/projects/a964bc2e302b4652ade830aa5d87c2d9/users/957c16574b464b49980f686e6cc2a002/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.942227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9248fb0-1d21-4d44-abf5-76ea65308b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a964bc2e302b4652ade830aa5d87c2d9, user_id=957c16574b464b49980f686e6cc2a002, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.948639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9248fb0-1d21-4d44-abf5-76ea65308b7e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.951546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fed24e-15a5-4358-a59c-9fa929296f76 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:10.956115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:10.956772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.957089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.957373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.959294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 985/1954] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:10 2022] PUT /identity/v3/projects/a964bc2e302b4652ade830aa5d87c2d9/users/957c16574b464b49980f686e6cc2a002/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:10.970531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92b510-9891-4c15-ab14-7f97dcb14c73 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:10.971022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92b510-9891-4c15-ab14-7f97dcb14c73 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:10.971160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:10.971302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92b510-9891-4c15-ab14-7f97dcb14c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:10.972517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:10.983482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 970/1955] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:11.003363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9c92b510-9891-4c15-ab14-7f97dcb14c73 None None] MFA Rules not processed for user `957c16574b464b49980f686e6cc2a002`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:11.004585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9c92b510-9891-4c15-ab14-7f97dcb14c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:11.087809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 986/1956] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:11.101535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:11.102992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:11.161049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de237f-d460-44aa-9977-c75542c5519f service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:11.212144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb90d29-25d0-4e06-b85e-e5a3d635573d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:11.212801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90d29-25d0-4e06-b85e-e5a3d635573d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:11.213690 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90d29-25d0-4e06-b85e-e5a3d635573d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:11.213919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90d29-25d0-4e06-b85e-e5a3d635573d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:11.223161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb90d29-25d0-4e06-b85e-e5a3d635573d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:11.224630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb90d29-25d0-4e06-b85e-e5a3d635573d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:11.234166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 971/1957] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:11.265207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92b510-9891-4c15-ab14-7f97dcb14c73 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:11.266649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9c92b510-9891-4c15-ab14-7f97dcb14c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:11.353629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92b510-9891-4c15-ab14-7f97dcb14c73 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:11.358345 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:11.359215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:11.359543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:11.359822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] PATH_INFO: `/v3/projects/2fe3f1e80cad48f29980ad57654aeae8` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:11.361025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe3f1e80cad48f29980ad57654aeae8)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:11.365419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:11.417281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe3f1e80cad48f29980ad57654aeae8', 'request_id': 'req-db96a9ef-a12e-465b-ba40-48ab77420232'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.424269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe3f1e80cad48f29980ad57654aeae8', 'request_id': 'req-db96a9ef-a12e-465b-ba40-48ab77420232'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.425935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:11.439104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe3f1e80cad48f29980ad57654aeae8', 'request_id': 'req-db96a9ef-a12e-465b-ba40-48ab77420232'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.455492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] The token cache is being invalidate because project 2fe3f1e80cad48f29980ad57654aeae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:11.455934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.456679 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.459247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 987/1958] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:11 2022] DELETE /identity/v3/projects/2fe3f1e80cad48f29980ad57654aeae8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:11.464203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb90d29-25d0-4e06-b85e-e5a3d635573d service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:11.475815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb90d29-25d0-4e06-b85e-e5a3d635573d service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:11.540790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:11.541713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:11.541990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:11.542264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] PATH_INFO: `/v3/users/4a02baae8582483fab2f8ff9c85e5190` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:11.543542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a02baae8582483fab2f8ff9c85e5190)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:11.545616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:11.593407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:11.594917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:11.601596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:11.603925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a02baae8582483fab2f8ff9c85e5190'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.613123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a02baae8582483fab2f8ff9c85e5190'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.626345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a02baae8582483fab2f8ff9c85e5190'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.627023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a02baae8582483fab2f8ff9c85e5190'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.633835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a02baae8582483fab2f8ff9c85e5190'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:11.640680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 972/1959] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:11 2022] DELETE /identity/v3/users/4a02baae8582483fab2f8ff9c85e5190 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:11.680764 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96a9ef-a12e-465b-ba40-48ab77420232 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:11.757217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '957c16574b464b49980f686e6cc2a002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a964bc2e302b4652ade830aa5d87c2d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:11.757995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:11.758288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:11.758602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:11.759316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:11.759677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:11.762037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:11.777669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 988/1960] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:11.781904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:11.783487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:11.808478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf26f3d-4b9f-4bb0-9525-4752332f9224 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:11.813572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c11bd-a808-4bb8-9ac7-7588724770c9 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '957c16574b464b49980f686e6cc2a002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a964bc2e302b4652ade830aa5d87c2d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:11.814229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c11bd-a808-4bb8-9ac7-7588724770c9 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:11.814516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c11bd-a808-4bb8-9ac7-7588724770c9 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:11.814794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c11bd-a808-4bb8-9ac7-7588724770c9 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:11.815533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c11bd-a808-4bb8-9ac7-7588724770c9 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:11.817509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c11bd-a808-4bb8-9ac7-7588724770c9 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:11.850202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 973/1961] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:11 2022] POST /identity/v3/projects => generated 354 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:11.851490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:11.867622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbed7cc-d2a5-4e5f-a985-e66886555e66 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:11.946011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '957c16574b464b49980f686e6cc2a002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a964bc2e302b4652ade830aa5d87c2d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:11.946879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:11.947166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:11.947696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:11.948975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:11.951404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:11.954997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:11.969616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 989/1962] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:12.012310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c11bd-a808-4bb8-9ac7-7588724770c9 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.017755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.020151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:12.025785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c11bd-a808-4bb8-9ac7-7588724770c9 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.030857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '957c16574b464b49980f686e6cc2a002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a964bc2e302b4652ade830aa5d87c2d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.031778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.032065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.032344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] PATH_INFO: `/v3/projects/c2ede471538947bc899a58da6f965849` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.033539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c2ede471538947bc899a58da6f965849)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.036851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.046722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:12.106345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ede471538947bc899a58da6f965849', 'request_id': 'req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.110421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c45976-f510-43bd-a0c9-943d4fed1607 tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.118657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ede471538947bc899a58da6f965849', 'request_id': 'req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.128073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ede471538947bc899a58da6f965849', 'request_id': 'req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.140182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] The token cache is being invalidate because project c2ede471538947bc899a58da6f965849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:12.140864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.141698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09aa6ebb-0018-4110-8aa8-c2b50c02f5da tempest-VolumeQuotaClassesTest-86478900 tempest-VolumeQuotaClassesTest-86478900-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.143786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 974/1963] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:12 2022] DELETE /identity/v3/projects/c2ede471538947bc899a58da6f965849 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:12.201616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.202665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.203186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.203837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] PATH_INFO: `/v3/projects/8c171927891346b688d93e772428e839` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.205473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c171927891346b688d93e772428e839)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.207438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.257274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c171927891346b688d93e772428e839', 'request_id': 'req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.264775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c171927891346b688d93e772428e839', 'request_id': 'req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.271716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c171927891346b688d93e772428e839', 'request_id': 'req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.284883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] The token cache is being invalidate because project 8c171927891346b688d93e772428e839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:12.285170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.285810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.287411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 990/1964] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:12 2022] DELETE /identity/v3/projects/8c171927891346b688d93e772428e839 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:12.295544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68cd0d-14f5-4137-8610-d5d8167a17bf None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.295544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68cd0d-14f5-4137-8610-d5d8167a17bf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.295544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68cd0d-14f5-4137-8610-d5d8167a17bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.313343 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-dd68cd0d-14f5-4137-8610-d5d8167a17bf None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:12.314568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dd68cd0d-14f5-4137-8610-d5d8167a17bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:12.366798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.368220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:12.377176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 975/1965] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:12.385466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91996a1f-6775-446a-ac18-c935e5ff447c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.385716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91996a1f-6775-446a-ac18-c935e5ff447c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.386003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91996a1f-6775-446a-ac18-c935e5ff447c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.408761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-91996a1f-6775-446a-ac18-c935e5ff447c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:12.410072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-91996a1f-6775-446a-ac18-c935e5ff447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:12.421787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 976/1966] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:12.425670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91996a1f-6775-446a-ac18-c935e5ff447c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.435909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91996a1f-6775-446a-ac18-c935e5ff447c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.437442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:12.441009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.441784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.442157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.442437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.443160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:12.443498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.443818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc69eaf-0d5c-423f-bcc6-b63dd0aab605 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.445099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.451044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.451965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.452244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.452885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] PATH_INFO: `/v3/users/4c559a715c7a4d3e8e28817cdc74ec9b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.453869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c559a715c7a4d3e8e28817cdc74ec9b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.455929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.457531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 977/1967] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:12.462787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.469871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:12.473615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfaab7-178c-4254-a21d-16f22fdc429b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.479606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.480252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.480542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.481068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.481927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.484241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.507165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c559a715c7a4d3e8e28817cdc74ec9b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.508596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c559a715c7a4d3e8e28817cdc74ec9b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.521396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 978/1968] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:12 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:12.521577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c559a715c7a4d3e8e28817cdc74ec9b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.522387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.525129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:12.534904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c559a715c7a4d3e8e28817cdc74ec9b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.535643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f095f2e4-5784-49f1-8491-24c91e710531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c559a715c7a4d3e8e28817cdc74ec9b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:12.550881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 991/1969] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:12 2022] DELETE /identity/v3/users/4c559a715c7a4d3e8e28817cdc74ec9b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:12.555033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a680-91b1-408e-bb8d-d465c2bd0ebb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.555322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a680-91b1-408e-bb8d-d465c2bd0ebb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.555633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a680-91b1-408e-bb8d-d465c2bd0ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.577176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5d00a680-91b1-408e-bb8d-d465c2bd0ebb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:12.578481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5d00a680-91b1-408e-bb8d-d465c2bd0ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:12.606090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84924a7-a6b8-4367-bbaf-93b2ec780985 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.647173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 992/1970] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:12.656882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00a680-91b1-408e-bb8d-d465c2bd0ebb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.658218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5d00a680-91b1-408e-bb8d-d465c2bd0ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:12.672263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d603064-b5e5-4cdd-8f16-74b3f439f8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.673112 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d603064-b5e5-4cdd-8f16-74b3f439f8bd admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.673612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d603064-b5e5-4cdd-8f16-74b3f439f8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.673993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d603064-b5e5-4cdd-8f16-74b3f439f8bd admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.674782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d603064-b5e5-4cdd-8f16-74b3f439f8bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.676851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d603064-b5e5-4cdd-8f16-74b3f439f8bd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.699052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 979/1971] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:12 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:12.702854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d603064-b5e5-4cdd-8f16-74b3f439f8bd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.712376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d603064-b5e5-4cdd-8f16-74b3f439f8bd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.716147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00a680-91b1-408e-bb8d-d465c2bd0ebb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.717274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a56ec-ab2f-4ef7-9f7d-3e41981721d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.717932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a56ec-ab2f-4ef7-9f7d-3e41981721d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.718213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a56ec-ab2f-4ef7-9f7d-3e41981721d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.718501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a56ec-ab2f-4ef7-9f7d-3e41981721d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.719382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a56ec-ab2f-4ef7-9f7d-3e41981721d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.720787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e662de-b48d-4b2c-9eda-0be8ac160856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.721001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a56ec-ab2f-4ef7-9f7d-3e41981721d6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.721903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e662de-b48d-4b2c-9eda-0be8ac160856 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.722349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e662de-b48d-4b2c-9eda-0be8ac160856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.722624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e662de-b48d-4b2c-9eda-0be8ac160856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.732799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e662de-b48d-4b2c-9eda-0be8ac160856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.733656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 980/1972] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:12.734366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e662de-b48d-4b2c-9eda-0be8ac160856 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.736792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a56ec-ab2f-4ef7-9f7d-3e41981721d6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.747566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a56ec-ab2f-4ef7-9f7d-3e41981721d6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.747767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 993/1973] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:26:12.752213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c24cd6-abe5-4c0f-84e5-040212939a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.752871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c24cd6-abe5-4c0f-84e5-040212939a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.753157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c24cd6-abe5-4c0f-84e5-040212939a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.753438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c24cd6-abe5-4c0f-84e5-040212939a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.754166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c24cd6-abe5-4c0f-84e5-040212939a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.755802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c24cd6-abe5-4c0f-84e5-040212939a2e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.766494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 981/1974] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:12.770567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e662de-b48d-4b2c-9eda-0be8ac160856 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:12.780335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e662de-b48d-4b2c-9eda-0be8ac160856 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:12.785413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ad2e-89f8-4ff2-977d-15d6d9f1fc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:12.786613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ad2e-89f8-4ff2-977d-15d6d9f1fc0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.786992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ad2e-89f8-4ff2-977d-15d6d9f1fc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.787405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ad2e-89f8-4ff2-977d-15d6d9f1fc0d admin admin] PATH_INFO: `/v3/projects/3aa0f2c25a0346d5802024c87bd9ea87/users/d535766c5aea4e809ca3d4df5db4bc96/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.804430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3ad2e-89f8-4ff2-977d-15d6d9f1fc0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa0f2c25a0346d5802024c87bd9ea87, user_id=d535766c5aea4e809ca3d4df5db4bc96, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:12.808138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3ad2e-89f8-4ff2-977d-15d6d9f1fc0d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:12.810895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:12.825298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 994/1975] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:12 2022] PUT /identity/v3/projects/3aa0f2c25a0346d5802024c87bd9ea87/users/d535766c5aea4e809ca3d4df5db4bc96/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:12.836000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154e3b-5164-438e-8cde-9da389bd1acf None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:12.836289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154e3b-5164-438e-8cde-9da389bd1acf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:12.836571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154e3b-5164-438e-8cde-9da389bd1acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:12.864902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a4154e3b-5164-438e-8cde-9da389bd1acf None None] MFA Rules not processed for user `d535766c5aea4e809ca3d4df5db4bc96`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:12.866493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a4154e3b-5164-438e-8cde-9da389bd1acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:12.920229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:12.943281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 982/1976] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:12.947813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ad2e-89f8-4ff2-977d-15d6d9f1fc0d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.042694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4154e3b-5164-438e-8cde-9da389bd1acf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.043267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ad2e-89f8-4ff2-977d-15d6d9f1fc0d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.045545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a4154e3b-5164-438e-8cde-9da389bd1acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.048287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750429ac-1061-4dee-8c49-0196decdc4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.048965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750429ac-1061-4dee-8c49-0196decdc4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.049250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750429ac-1061-4dee-8c49-0196decdc4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.049531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750429ac-1061-4dee-8c49-0196decdc4a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.050270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750429ac-1061-4dee-8c49-0196decdc4a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.052789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750429ac-1061-4dee-8c49-0196decdc4a4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.065186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4154e3b-5164-438e-8cde-9da389bd1acf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.073013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.075955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.076250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.076578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] PATH_INFO: `/v3/projects/8ec9d15a03184e2c908f97e69a0736f6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.078214 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec9d15a03184e2c908f97e69a0736f6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.081019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.104940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 995/1977] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:12 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:13.109128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750429ac-1061-4dee-8c49-0196decdc4a4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.138595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec9d15a03184e2c908f97e69a0736f6', 'request_id': 'req-87548bab-a385-40d3-9db3-a86b1c0c24ca'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.147732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec9d15a03184e2c908f97e69a0736f6', 'request_id': 'req-87548bab-a385-40d3-9db3-a86b1c0c24ca'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.155608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec9d15a03184e2c908f97e69a0736f6', 'request_id': 'req-87548bab-a385-40d3-9db3-a86b1c0c24ca'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.169810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] The token cache is being invalidate because project 8ec9d15a03184e2c908f97e69a0736f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:13.170097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.170946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.172776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 983/1978] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:13 2022] DELETE /identity/v3/projects/8ec9d15a03184e2c908f97e69a0736f6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:13.177251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.178721 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.189426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750429ac-1061-4dee-8c49-0196decdc4a4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.260888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87548bab-a385-40d3-9db3-a86b1c0c24ca admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.267862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.269168 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.269553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.270320 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] PATH_INFO: `/v3/users/957c16574b464b49980f686e6cc2a002` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.271865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=957c16574b464b49980f686e6cc2a002)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.274654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.287439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.288128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.288481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.288844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.289606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.292684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.319377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 996/1979] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:13 2022] POST /identity/v3/users => generated 298 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:13.322614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.325207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.338422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791871de-ca73-412d-a855-d1067c7ad377 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.338910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957c16574b464b49980f686e6cc2a002'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.344116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd193c-ef6c-4ba8-b997-fc09cf21a344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.345109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd193c-ef6c-4ba8-b997-fc09cf21a344 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.345532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd193c-ef6c-4ba8-b997-fc09cf21a344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.346046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd193c-ef6c-4ba8-b997-fc09cf21a344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.346902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd193c-ef6c-4ba8-b997-fc09cf21a344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.348911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd193c-ef6c-4ba8-b997-fc09cf21a344 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.349406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957c16574b464b49980f686e6cc2a002'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.361270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 997/1980] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:13.365517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd193c-ef6c-4ba8-b997-fc09cf21a344 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.367848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957c16574b464b49980f686e6cc2a002'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.368593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957c16574b464b49980f686e6cc2a002'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.371286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:13.373981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:13.378963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd193c-ef6c-4ba8-b997-fc09cf21a344 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.380678 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957c16574b464b49980f686e6cc2a002'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.385190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a25b4-c55c-4eb9-8d12-2174e8219b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.386163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a25b4-c55c-4eb9-8d12-2174e8219b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.386369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a25b4-c55c-4eb9-8d12-2174e8219b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.386645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a25b4-c55c-4eb9-8d12-2174e8219b01 admin admin] PATH_INFO: `/v3/projects/c8c1ac5a8089436195318039c091702f/users/4db7f064259c4a93bff48bda8c747c62/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.388623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 984/1981] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:13 2022] DELETE /identity/v3/users/957c16574b464b49980f686e6cc2a002 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:13.402038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a25b4-c55c-4eb9-8d12-2174e8219b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c1ac5a8089436195318039c091702f, user_id=4db7f064259c4a93bff48bda8c747c62, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.404784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a25b4-c55c-4eb9-8d12-2174e8219b01 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.421046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 998/1982] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:13 2022] PUT /identity/v3/projects/c8c1ac5a8089436195318039c091702f/users/4db7f064259c4a93bff48bda8c747c62/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:13.425207 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.426692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.499880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5d75-f483-4060-b940-803a91aeb5c7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.572349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3be1a6-126e-4ac0-a335-d493743ea0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.573692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3be1a6-126e-4ac0-a335-d493743ea0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.573692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3be1a6-126e-4ac0-a335-d493743ea0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.573692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3be1a6-126e-4ac0-a335-d493743ea0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.574363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3be1a6-126e-4ac0-a335-d493743ea0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.576050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3be1a6-126e-4ac0-a335-d493743ea0e0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.586658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 985/1983] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:13.591620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a25b4-c55c-4eb9-8d12-2174e8219b01 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.602904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a25b4-c55c-4eb9-8d12-2174e8219b01 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.608166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.609187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.609539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.609882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] PATH_INFO: `/v3/projects/c8c1ac5a8089436195318039c091702f/users/4db7f064259c4a93bff48bda8c747c62/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.611970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c1ac5a8089436195318039c091702f, user_id=4db7f064259c4a93bff48bda8c747c62, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.614689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.628363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 999/1984] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:13 2022] PUT /identity/v3/projects/c8c1ac5a8089436195318039c091702f/users/4db7f064259c4a93bff48bda8c747c62/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:13.639445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f132f9-6eb1-4288-8bc9-7c04d645fc52 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.639942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f132f9-6eb1-4288-8bc9-7c04d645fc52 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.640423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f132f9-6eb1-4288-8bc9-7c04d645fc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.689241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d4f132f9-6eb1-4288-8bc9-7c04d645fc52 None None] MFA Rules not processed for user `4db7f064259c4a93bff48bda8c747c62`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:13.690552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d4f132f9-6eb1-4288-8bc9-7c04d645fc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.719404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.723475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.796296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09837ef6-e40b-4fe2-beb5-02a5e50a71c8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.801462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.802361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.802474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 986/1985] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:13.802664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.802918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] PATH_INFO: `/v3/projects/a964bc2e302b4652ade830aa5d87c2d9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.804145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a964bc2e302b4652ade830aa5d87c2d9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:13.806060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:13.813902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f132f9-6eb1-4288-8bc9-7c04d645fc52 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:13.816404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d4f132f9-6eb1-4288-8bc9-7c04d645fc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.837228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:13.860457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a964bc2e302b4652ade830aa5d87c2d9', 'request_id': 'req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.870849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a964bc2e302b4652ade830aa5d87c2d9', 'request_id': 'req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.879172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a964bc2e302b4652ade830aa5d87c2d9', 'request_id': 'req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.890331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f132f9-6eb1-4288-8bc9-7c04d645fc52 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:13.896105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] The token cache is being invalidate because project a964bc2e302b4652ade830aa5d87c2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:13.896405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.897207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-24471e42-0c3c-40ea-9fcc-f5ebb50e68d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:13.898791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1000/1986] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:13 2022] DELETE /identity/v3/projects/a964bc2e302b4652ade830aa5d87c2d9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:13.905896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296c680-739d-4009-8c06-2c563b91d940 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.906231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296c680-739d-4009-8c06-2c563b91d940 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.906441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296c680-739d-4009-8c06-2c563b91d940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.930829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0296c680-739d-4009-8c06-2c563b91d940 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:13.932313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0296c680-739d-4009-8c06-2c563b91d940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.954203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:13.954991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:13.955306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:13.955598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:13.957400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:13.997205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:14.011388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1001/1987] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:14.023125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b84a-89fc-4a76-9d93-f39848ed0882 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.023125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b84a-89fc-4a76-9d93-f39848ed0882 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.023125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b84a-89fc-4a76-9d93-f39848ed0882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.040658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c494b84a-89fc-4a76-9d93-f39848ed0882 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:14.042037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.042431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c494b84a-89fc-4a76-9d93-f39848ed0882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:14.043534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.050644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1002/1988] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:14.054545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494b84a-89fc-4a76-9d93-f39848ed0882 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.062470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 987/1989] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:14.063991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494b84a-89fc-4a76-9d93-f39848ed0882 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.069046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.070053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.070356 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.070777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.071701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:14.072051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.075986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.089346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1003/1990] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:26:14.093650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.112698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1f862-2ae9-4bb6-b228-128c119aad3d service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.120522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.121542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.121927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.122304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.123506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.125724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.164255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 988/1991] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:14 2022] POST /identity/v3/projects => generated 399 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:14.168358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.180180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38fd72-dd14-458d-8f0f-105fba1350c8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.205018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.212056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:14.279300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4aa17-95fb-44fe-83a3-a9831904de2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.281280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4aa17-95fb-44fe-83a3-a9831904de2d admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.281974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4aa17-95fb-44fe-83a3-a9831904de2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.281974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4aa17-95fb-44fe-83a3-a9831904de2d admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.282812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4aa17-95fb-44fe-83a3-a9831904de2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.286383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4aa17-95fb-44fe-83a3-a9831904de2d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.302833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc39e5-3778-487e-ae5a-f1126dbc1c52 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.309047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519058a9-bbf9-4ab4-a0b0-2bdc8e9668f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.309720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519058a9-bbf9-4ab4-a0b0-2bdc8e9668f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.310004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519058a9-bbf9-4ab4-a0b0-2bdc8e9668f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.310281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519058a9-bbf9-4ab4-a0b0-2bdc8e9668f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.312881 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-519058a9-bbf9-4ab4-a0b0-2bdc8e9668f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:14.331284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1004/1992] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:14 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:14.333236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4aa17-95fb-44fe-83a3-a9831904de2d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.337671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:14.347364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4aa17-95fb-44fe-83a3-a9831904de2d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.352610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab48c7a-ea66-444f-bff4-f7cf6ab284f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.353528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48c7a-ea66-444f-bff4-f7cf6ab284f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.353840 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48c7a-ea66-444f-bff4-f7cf6ab284f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.354580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48c7a-ea66-444f-bff4-f7cf6ab284f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.355099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab48c7a-ea66-444f-bff4-f7cf6ab284f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.356898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab48c7a-ea66-444f-bff4-f7cf6ab284f5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.368483 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1005/1993] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:14.372774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab48c7a-ea66-444f-bff4-f7cf6ab284f5 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.401690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab48c7a-ea66-444f-bff4-f7cf6ab284f5 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.413749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adcbd1-dac5-43ea-8302-d75095a3da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.414848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adcbd1-dac5-43ea-8302-d75095a3da0b admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.415309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adcbd1-dac5-43ea-8302-d75095a3da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.415873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adcbd1-dac5-43ea-8302-d75095a3da0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.417089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adcbd1-dac5-43ea-8302-d75095a3da0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.419552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adcbd1-dac5-43ea-8302-d75095a3da0b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.427520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519058a9-bbf9-4ab4-a0b0-2bdc8e9668f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.429112 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519058a9-bbf9-4ab4-a0b0-2bdc8e9668f4 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.436789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1006/1994] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:14.437869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adcbd1-dac5-43ea-8302-d75095a3da0b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.442275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 989/1995] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:26:14.450523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adcbd1-dac5-43ea-8302-d75095a3da0b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.457759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf541d-ffb6-464b-b9b4-71af997ba3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.459181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf541d-ffb6-464b-b9b4-71af997ba3f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.459690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf541d-ffb6-464b-b9b4-71af997ba3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.460133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf541d-ffb6-464b-b9b4-71af997ba3f6 admin admin] PATH_INFO: `/v3/projects/a17f25cc9a2d4fe78900d2aefc296a6c/users/7a2f8e7fe3f1440090d72592cd4e3c38/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.482491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf541d-ffb6-464b-b9b4-71af997ba3f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17f25cc9a2d4fe78900d2aefc296a6c, user_id=7a2f8e7fe3f1440090d72592cd4e3c38, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.485789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf541d-ffb6-464b-b9b4-71af997ba3f6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.506307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1007/1996] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:14 2022] PUT /identity/v3/projects/a17f25cc9a2d4fe78900d2aefc296a6c/users/7a2f8e7fe3f1440090d72592cd4e3c38/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:14.517627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818a6c5-c9d4-4805-9b69-3f778f6e9251 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.517918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818a6c5-c9d4-4805-9b69-3f778f6e9251 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.518204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818a6c5-c9d4-4805-9b69-3f778f6e9251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.559131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e818a6c5-c9d4-4805-9b69-3f778f6e9251 None None] MFA Rules not processed for user `7a2f8e7fe3f1440090d72592cd4e3c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:14.560324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e818a6c5-c9d4-4805-9b69-3f778f6e9251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:14.642386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 990/1997] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:14.646819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf541d-ffb6-464b-b9b4-71af997ba3f6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.721072 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf541d-ffb6-464b-b9b4-71af997ba3f6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.728446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ed35d-4de7-4fa6-a99d-c8da3e804479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.730306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ed35d-4de7-4fa6-a99d-c8da3e804479 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.730594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ed35d-4de7-4fa6-a99d-c8da3e804479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.730872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ed35d-4de7-4fa6-a99d-c8da3e804479 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.731690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ed35d-4de7-4fa6-a99d-c8da3e804479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.734004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ed35d-4de7-4fa6-a99d-c8da3e804479 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.804407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1008/1998] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:14 2022] POST /identity/v3/projects => generated 401 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:14.809720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818a6c5-c9d4-4805-9b69-3f778f6e9251 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.836934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818a6c5-c9d4-4805-9b69-3f778f6e9251 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.912734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd6dbc-93d4-42e5-9d7a-cf73e7deab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.913407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd6dbc-93d4-42e5-9d7a-cf73e7deab3a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.913685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd6dbc-93d4-42e5-9d7a-cf73e7deab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.913959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd6dbc-93d4-42e5-9d7a-cf73e7deab3a admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.914643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd6dbc-93d4-42e5-9d7a-cf73e7deab3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.916710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd6dbc-93d4-42e5-9d7a-cf73e7deab3a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.935305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 991/1999] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:14 2022] POST /identity/v3/users => generated 307 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:14.939368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ed35d-4de7-4fa6-a99d-c8da3e804479 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.950737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ed35d-4de7-4fa6-a99d-c8da3e804479 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.955499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd3384-f5ad-454b-b456-81b302cda2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.956191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3384-f5ad-454b-b456-81b302cda2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.956477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3384-f5ad-454b-b456-81b302cda2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.956780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3384-f5ad-454b-b456-81b302cda2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:14.957537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd3384-f5ad-454b-b456-81b302cda2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:14.959526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd3384-f5ad-454b-b456-81b302cda2a5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:14.960805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:14.974770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1009/2000] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:14.976684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd6dbc-93d4-42e5-9d7a-cf73e7deab3a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:14.986540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd6dbc-93d4-42e5-9d7a-cf73e7deab3a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:14.991021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1550e-b1dd-4479-8c2b-f896ba4720ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:14.991920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1550e-b1dd-4479-8c2b-f896ba4720ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:14.992203 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1550e-b1dd-4479-8c2b-f896ba4720ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:14.992476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1550e-b1dd-4479-8c2b-f896ba4720ce admin admin] PATH_INFO: `/v3/projects/0f9ac503607545a59356709e9dfa80be/users/f1910c9c51a84a2f9f7c2c720c22f472/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:15.007869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1550e-b1dd-4479-8c2b-f896ba4720ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9ac503607545a59356709e9dfa80be, user_id=f1910c9c51a84a2f9f7c2c720c22f472, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:15.011428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1550e-b1dd-4479-8c2b-f896ba4720ce admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:15.023900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 992/2001] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:14 2022] PUT /identity/v3/projects/0f9ac503607545a59356709e9dfa80be/users/f1910c9c51a84a2f9f7c2c720c22f472/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:15.028684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd3384-f5ad-454b-b456-81b302cda2a5 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:15.038531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd3384-f5ad-454b-b456-81b302cda2a5 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:15.094334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8684-2647-4be1-848b-50ca85d882bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:15.094907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8684-2647-4be1-848b-50ca85d882bd admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:15.095183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8684-2647-4be1-848b-50ca85d882bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:15.095489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8684-2647-4be1-848b-50ca85d882bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:15.096206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d8684-2647-4be1-848b-50ca85d882bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:15.097792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d8684-2647-4be1-848b-50ca85d882bd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:15.109311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1010/2002] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:15.113478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1550e-b1dd-4479-8c2b-f896ba4720ce admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:15.124139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1550e-b1dd-4479-8c2b-f896ba4720ce admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:15.128724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240d590-6efe-4d36-a773-a532865ce5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:15.129661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d590-6efe-4d36-a773-a532865ce5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:15.129945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d590-6efe-4d36-a773-a532865ce5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:15.130228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d590-6efe-4d36-a773-a532865ce5fa admin admin] PATH_INFO: `/v3/projects/0f9ac503607545a59356709e9dfa80be/users/f1910c9c51a84a2f9f7c2c720c22f472/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:15.132235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240d590-6efe-4d36-a773-a532865ce5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9ac503607545a59356709e9dfa80be, user_id=f1910c9c51a84a2f9f7c2c720c22f472, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:15.135742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240d590-6efe-4d36-a773-a532865ce5fa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:15.142409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:15.148713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 993/2003] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:15 2022] PUT /identity/v3/projects/0f9ac503607545a59356709e9dfa80be/users/f1910c9c51a84a2f9f7c2c720c22f472/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:15.159911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948ef9-e92a-4dec-8b3d-4ada2c663853 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:15.160211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948ef9-e92a-4dec-8b3d-4ada2c663853 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:15.160516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948ef9-e92a-4dec-8b3d-4ada2c663853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:15.191529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c3948ef9-e92a-4dec-8b3d-4ada2c663853 None None] MFA Rules not processed for user `f1910c9c51a84a2f9f7c2c720c22f472`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:15.192693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c3948ef9-e92a-4dec-8b3d-4ada2c663853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:15.280753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1011/2004] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:15.292020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240d590-6efe-4d36-a773-a532865ce5fa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:15.372749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240d590-6efe-4d36-a773-a532865ce5fa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:15.376572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:15.377361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:15.377736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:15.378109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:15.388842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:15.390267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:15.400484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 994/2005] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:15.478354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3948ef9-e92a-4dec-8b3d-4ada2c663853 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:15.488633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3948ef9-e92a-4dec-8b3d-4ada2c663853 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:15.501634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de03dd-43a9-454c-ab87-30a0eeb15002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:15.502304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de03dd-43a9-454c-ab87-30a0eeb15002 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:15.502589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de03dd-43a9-454c-ab87-30a0eeb15002 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:15.502867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de03dd-43a9-454c-ab87-30a0eeb15002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:15.569795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de03dd-43a9-454c-ab87-30a0eeb15002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:15.571230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de03dd-43a9-454c-ab87-30a0eeb15002 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:15.580652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1012/2006] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:16.591316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:16.592833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:16.674750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279f51f-d022-4970-ba83-558188c57c01 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:16.679508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:16.680403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:16.680822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:16.681108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] PATH_INFO: `/v3/users/8c2297338af04003a66d2bdb7a644f99` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:16.682855 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c2297338af04003a66d2bdb7a644f99)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:16.684972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:16.739591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c2297338af04003a66d2bdb7a644f99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:16.747642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c2297338af04003a66d2bdb7a644f99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:16.760704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c2297338af04003a66d2bdb7a644f99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:16.761356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c2297338af04003a66d2bdb7a644f99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:16.768486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c2297338af04003a66d2bdb7a644f99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:16.774963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 995/2007] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:16 2022] DELETE /identity/v3/users/8c2297338af04003a66d2bdb7a644f99 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:16.779781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91056fcf-4227-4982-864f-cffe305f2e3f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:16.780344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91056fcf-4227-4982-864f-cffe305f2e3f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:16.780344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91056fcf-4227-4982-864f-cffe305f2e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:16.798272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-91056fcf-4227-4982-864f-cffe305f2e3f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:16.799329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-91056fcf-4227-4982-864f-cffe305f2e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:16.810635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:16.868266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1013/2008] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:16.879473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:16.880910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:16.903696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:16.942221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c10e0-3886-4d8b-b185-fba31ddd78bf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:16.946042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c649034-1835-4e7e-ba54-1dcfd53c2d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:16.948049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c649034-1835-4e7e-ba54-1dcfd53c2d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:16.948563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c649034-1835-4e7e-ba54-1dcfd53c2d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:16.948913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c649034-1835-4e7e-ba54-1dcfd53c2d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:16.959095 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c649034-1835-4e7e-ba54-1dcfd53c2d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:16.960589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c649034-1835-4e7e-ba54-1dcfd53c2d0a service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:16.970495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 996/2009] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:26:17.387502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91056fcf-4227-4982-864f-cffe305f2e3f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:17.388715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-91056fcf-4227-4982-864f-cffe305f2e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:17.402186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91056fcf-4227-4982-864f-cffe305f2e3f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:17.413403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:17.414210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:17.414491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:17.422739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] PATH_INFO: `/v3/projects/65311989b976446ca8e8822952306bf4` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:17.424007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=65311989b976446ca8e8822952306bf4)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:17.425934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:17.479890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65311989b976446ca8e8822952306bf4', 'request_id': 'req-75a0e403-fbf9-4e53-b828-cdbb5d382eda'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.485997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65311989b976446ca8e8822952306bf4', 'request_id': 'req-75a0e403-fbf9-4e53-b828-cdbb5d382eda'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.491563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65311989b976446ca8e8822952306bf4', 'request_id': 'req-75a0e403-fbf9-4e53-b828-cdbb5d382eda'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.504998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] The token cache is being invalidate because project 65311989b976446ca8e8822952306bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:17.505279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.505889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.507574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1014/2010] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:17 2022] DELETE /identity/v3/projects/65311989b976446ca8e8822952306bf4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:17.512204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c649034-1835-4e7e-ba54-1dcfd53c2d0a service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:17.522419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c649034-1835-4e7e-ba54-1dcfd53c2d0a service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:17.583433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:17.584322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:17.584674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:17.584955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] PATH_INFO: `/v3/users/d733141c7a834049bb5e769cc4b5f934` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:17.586237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d733141c7a834049bb5e769cc4b5f934)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:17.588251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:17.635694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd733141c7a834049bb5e769cc4b5f934'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.643607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd733141c7a834049bb5e769cc4b5f934'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.656847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd733141c7a834049bb5e769cc4b5f934'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.657601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd733141c7a834049bb5e769cc4b5f934'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.664742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5da211a0-fdcb-4d59-bf88-9bc45b73b0cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd733141c7a834049bb5e769cc4b5f934'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:17.670710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 997/2011] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:17 2022] DELETE /identity/v3/users/d733141c7a834049bb5e769cc4b5f934 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:18.029387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:18.033042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:18.053975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0e403-fbf9-4e53-b828-cdbb5d382eda admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:18.132176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:18.133178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.133459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.133736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] PATH_INFO: `/v3/projects/36de2dcc992e4733885ccb064d22d81a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.136009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] RBAC: Authorizing `identity:delete_project(project_id=36de2dcc992e4733885ccb064d22d81a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:18.138149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:18.198794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:18.210621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36de2dcc992e4733885ccb064d22d81a', 'request_id': 'req-69fd846d-0790-4f0b-877b-475dfcea260a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.220260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36de2dcc992e4733885ccb064d22d81a', 'request_id': 'req-69fd846d-0790-4f0b-877b-475dfcea260a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.228095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36de2dcc992e4733885ccb064d22d81a', 'request_id': 'req-69fd846d-0790-4f0b-877b-475dfcea260a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.245510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] The token cache is being invalidate because project 36de2dcc992e4733885ccb064d22d81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:18.245944 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.246727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.248625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1015/2012] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:18 2022] DELETE /identity/v3/projects/36de2dcc992e4733885ccb064d22d81a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:18.254959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212497f-8082-4ecd-b461-faa314db4590 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.255252 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212497f-8082-4ecd-b461-faa314db4590 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.255631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212497f-8082-4ecd-b461-faa314db4590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.280578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d212497f-8082-4ecd-b461-faa314db4590 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:18.281810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d212497f-8082-4ecd-b461-faa314db4590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:18.342001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:18.375799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 998/2013] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:18.381284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:18.383637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:18.395181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7344-c5ec-465b-93c0-816f7f026c14 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.398331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7344-c5ec-465b-93c0-816f7f026c14 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.398648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7344-c5ec-465b-93c0-816f7f026c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.425540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd846d-0790-4f0b-877b-475dfcea260a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:18.439192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cc1d7344-c5ec-465b-93c0-816f7f026c14 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:18.443394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:18.444572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cc1d7344-c5ec-465b-93c0-816f7f026c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:18.444824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.445301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.445794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] PATH_INFO: `/v3/users/d535766c5aea4e809ca3d4df5db4bc96` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.457582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 999/2014] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:18.468938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d535766c5aea4e809ca3d4df5db4bc96)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:18.471279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:18.472133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d7344-c5ec-465b-93c0-816f7f026c14 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:18.486825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d7344-c5ec-465b-93c0-816f7f026c14 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:18.493286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:18.494090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.496046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.496333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.497089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:18.497456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:18.499148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:18.520663 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1000/2015] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:26:18.521679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:18.538377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335d1b-f87a-4cf3-b5c6-3b6d0d26a5c7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:18.546985 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:18.549154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.551405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.552022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.553025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:18.556686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:18.593501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd535766c5aea4e809ca3d4df5db4bc96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.593501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd535766c5aea4e809ca3d4df5db4bc96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.607398 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd535766c5aea4e809ca3d4df5db4bc96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.618962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1001/2016] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:18 2022] POST /identity/v3/projects => generated 359 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:18.624175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:18.628870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:18.638443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd535766c5aea4e809ca3d4df5db4bc96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.639303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8fe4a621-1727-4053-b349-fce164d5cff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd535766c5aea4e809ca3d4df5db4bc96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:18.662348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1016/2017] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:18 2022] DELETE /identity/v3/users/d535766c5aea4e809ca3d4df5db4bc96 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:18.664231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c8d3-336c-48ff-ba3c-a08e43b70f23 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.664641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c8d3-336c-48ff-ba3c-a08e43b70f23 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.664990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c8d3-336c-48ff-ba3c-a08e43b70f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.700363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2454c8d3-336c-48ff-ba3c-a08e43b70f23 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:18.703306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2454c8d3-336c-48ff-ba3c-a08e43b70f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:18.783373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b7abc-6257-4361-b1b4-8089ce761dac admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:18.854085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1017/2018] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:18.863008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c8d3-336c-48ff-ba3c-a08e43b70f23 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:18.865299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2454c8d3-336c-48ff-ba3c-a08e43b70f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:18.870712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f95d04-db85-453e-948d-f8d1cb9a7bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:18.871496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f95d04-db85-453e-948d-f8d1cb9a7bcd admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.871897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f95d04-db85-453e-948d-f8d1cb9a7bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.872272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f95d04-db85-453e-948d-f8d1cb9a7bcd admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.873046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f95d04-db85-453e-948d-f8d1cb9a7bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:18.875536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f95d04-db85-453e-948d-f8d1cb9a7bcd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:18.904653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1002/2019] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:18 2022] POST /identity/v3/users => generated 280 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:18.909091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f95d04-db85-453e-948d-f8d1cb9a7bcd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:18.928085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f95d04-db85-453e-948d-f8d1cb9a7bcd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:18.939745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:18.940368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ba7b4-6908-473f-9541-39352bb635ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:18.942340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ba7b4-6908-473f-9541-39352bb635ed admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.942875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ba7b4-6908-473f-9541-39352bb635ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.943168 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ba7b4-6908-473f-9541-39352bb635ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.944558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ba7b4-6908-473f-9541-39352bb635ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:18.949194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ba7b4-6908-473f-9541-39352bb635ed admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:18.966811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1003/2020] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:18.969127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ba7b4-6908-473f-9541-39352bb635ed admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:18.978791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c8d3-336c-48ff-ba3c-a08e43b70f23 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:18.982689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ff3fb-0226-46da-9e67-e6cd21f215e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:18.983408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ff3fb-0226-46da-9e67-e6cd21f215e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.983754 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ff3fb-0226-46da-9e67-e6cd21f215e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.984030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ff3fb-0226-46da-9e67-e6cd21f215e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.987973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ba7b4-6908-473f-9541-39352bb635ed admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:18.993018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cef62e-6e45-406a-ba8d-84d5225f76ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:18.993697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef62e-6e45-406a-ba8d-84d5225f76ee admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:18.994557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef62e-6e45-406a-ba8d-84d5225f76ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:18.995119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef62e-6e45-406a-ba8d-84d5225f76ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:18.995897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cef62e-6e45-406a-ba8d-84d5225f76ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:18.996889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ff3fb-0226-46da-9e67-e6cd21f215e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:18.997579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cef62e-6e45-406a-ba8d-84d5225f76ee admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:18.998359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ff3fb-0226-46da-9e67-e6cd21f215e0 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:19.008645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1018/2021] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:19.021596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1004/2022] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:19.022723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ff3fb-0226-46da-9e67-e6cd21f215e0 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:19.033776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ff3fb-0226-46da-9e67-e6cd21f215e0 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:19.039362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:19.040359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:19.040643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:19.041570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] PATH_INFO: `/v3/projects/383eaf1769374b1e88af149c0907ad85/users/354193c494084abcbf27d428346e54e6/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:19.060250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=383eaf1769374b1e88af149c0907ad85, user_id=354193c494084abcbf27d428346e54e6, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:19.062991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:19.079212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1019/2023] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:19 2022] PUT /identity/v3/projects/383eaf1769374b1e88af149c0907ad85/users/354193c494084abcbf27d428346e54e6/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:19.091047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701a970-328a-45b8-bfc6-852eec8d1db4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:19.091414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701a970-328a-45b8-bfc6-852eec8d1db4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:19.091669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701a970-328a-45b8-bfc6-852eec8d1db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:19.124319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c701a970-328a-45b8-bfc6-852eec8d1db4 None None] MFA Rules not processed for user `354193c494084abcbf27d428346e54e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:19.125545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c701a970-328a-45b8-bfc6-852eec8d1db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:19.129981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:19.204582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1005/2024] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:19.237755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:19.239563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:19.298489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46589d-b3a4-40cd-b0a1-35949e5ca58e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:19.302497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45031-dabf-421e-8c5a-58e8473f3a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:19.303181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45031-dabf-421e-8c5a-58e8473f3a74 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:19.303498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45031-dabf-421e-8c5a-58e8473f3a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:19.303774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45031-dabf-421e-8c5a-58e8473f3a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:19.309511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701a970-328a-45b8-bfc6-852eec8d1db4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:19.311366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c701a970-328a-45b8-bfc6-852eec8d1db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:19.321584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe45031-dabf-421e-8c5a-58e8473f3a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:19.323946 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe45031-dabf-421e-8c5a-58e8473f3a74 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:19.341587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1020/2025] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:19 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:19.356852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45031-dabf-421e-8c5a-58e8473f3a74 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:19.398767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701a970-328a-45b8-bfc6-852eec8d1db4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:19.404503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:19.405386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:19.405669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:19.405946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] PATH_INFO: `/v3/projects/3aa0f2c25a0346d5802024c87bd9ea87` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:19.407158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa0f2c25a0346d5802024c87bd9ea87)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:19.409120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:19.438772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45031-dabf-421e-8c5a-58e8473f3a74 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:19.443685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:19.444561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:19.444837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:19.445110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] PATH_INFO: `/v3/users/354193c494084abcbf27d428346e54e6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:19.446359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=354193c494084abcbf27d428346e54e6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:19.452654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:19.465986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa0f2c25a0346d5802024c87bd9ea87', 'request_id': 'req-31378553-a0d1-4bff-a8f1-9025681ef187'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.474612 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa0f2c25a0346d5802024c87bd9ea87', 'request_id': 'req-31378553-a0d1-4bff-a8f1-9025681ef187'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.481272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa0f2c25a0346d5802024c87bd9ea87', 'request_id': 'req-31378553-a0d1-4bff-a8f1-9025681ef187'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.493442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] The token cache is being invalidate because project 3aa0f2c25a0346d5802024c87bd9ea87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:19.493732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.494426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.496300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1006/2026] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:19 2022] DELETE /identity/v3/projects/3aa0f2c25a0346d5802024c87bd9ea87 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:19.500674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:19.502100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:19.509010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354193c494084abcbf27d428346e54e6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.510240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354193c494084abcbf27d428346e54e6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.521968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354193c494084abcbf27d428346e54e6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.533586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:19.542275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354193c494084abcbf27d428346e54e6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.543453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c9a08e7f-7736-48a1-ad47-ea240e839394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354193c494084abcbf27d428346e54e6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.561616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1021/2027] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:19 2022] DELETE /identity/v3/users/354193c494084abcbf27d428346e54e6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:19.565724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:19.566227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c12a7-5627-4ba5-a2e2-44fdc8948eb2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:19.566533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c12a7-5627-4ba5-a2e2-44fdc8948eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:19.566889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c12a7-5627-4ba5-a2e2-44fdc8948eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:19.589374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-567c12a7-5627-4ba5-a2e2-44fdc8948eb2 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:19.590336 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-567c12a7-5627-4ba5-a2e2-44fdc8948eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:19.607109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31378553-a0d1-4bff-a8f1-9025681ef187 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:19.680122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:19.681079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:19.681327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:19.681892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] PATH_INFO: `/v3/users/4db7f064259c4a93bff48bda8c747c62` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:19.683255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db7f064259c4a93bff48bda8c747c62)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:19.685225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:19.700519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1022/2028] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:19.710383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c12a7-5627-4ba5-a2e2-44fdc8948eb2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:19.712222 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-567c12a7-5627-4ba5-a2e2-44fdc8948eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:19.750165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db7f064259c4a93bff48bda8c747c62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.759694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db7f064259c4a93bff48bda8c747c62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.781838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db7f064259c4a93bff48bda8c747c62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.782630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db7f064259c4a93bff48bda8c747c62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.790753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c12a7-5627-4ba5-a2e2-44fdc8948eb2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:19.792632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db7f064259c4a93bff48bda8c747c62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:19.795152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:19.795838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:19.796121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:19.796681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:19.798682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:19.799453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1007/2029] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:19 2022] DELETE /identity/v3/users/4db7f064259c4a93bff48bda8c747c62 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:19.818120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:19.819949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:19.830749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1023/2030] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:26:20.216923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.218698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.302242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09f570-ac44-4c8c-9595-533c94bccf07 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.309558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:20.310489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.310821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.311097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] PATH_INFO: `/v3/users/7a2f8e7fe3f1440090d72592cd4e3c38` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.312703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a2f8e7fe3f1440090d72592cd4e3c38)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:20.315015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:20.317444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:20.371314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.374423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.398951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85c513-e1b2-4672-8f44-826c4dc2a77e service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.407328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:20.408241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.408529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.408809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] PATH_INFO: `/v3/projects/c8c1ac5a8089436195318039c091702f` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.410389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c1ac5a8089436195318039c091702f)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:20.412587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:20.420758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a2f8e7fe3f1440090d72592cd4e3c38'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.429610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a2f8e7fe3f1440090d72592cd4e3c38'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.443255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a2f8e7fe3f1440090d72592cd4e3c38'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.443973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a2f8e7fe3f1440090d72592cd4e3c38'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.451970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a2f8e7fe3f1440090d72592cd4e3c38'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.457945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1008/2031] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:20 2022] DELETE /identity/v3/users/7a2f8e7fe3f1440090d72592cd4e3c38 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:20.460568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.462155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.469265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c1ac5a8089436195318039c091702f', 'request_id': 'req-611abc24-1ce1-4ba3-99c3-a39e4b53205e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.476611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c1ac5a8089436195318039c091702f', 'request_id': 'req-611abc24-1ce1-4ba3-99c3-a39e4b53205e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.485267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c1ac5a8089436195318039c091702f', 'request_id': 'req-611abc24-1ce1-4ba3-99c3-a39e4b53205e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.501026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] The token cache is being invalidate because project c8c1ac5a8089436195318039c091702f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:20.501642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.502462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-611abc24-1ce1-4ba3-99c3-a39e4b53205e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.507677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1024/2032] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:20 2022] DELETE /identity/v3/projects/c8c1ac5a8089436195318039c091702f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:20.507677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca446e78-c301-459d-9685-a1e51b3dd3eb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.507677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca446e78-c301-459d-9685-a1e51b3dd3eb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.507677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca446e78-c301-459d-9685-a1e51b3dd3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.528597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ca446e78-c301-459d-9685-a1e51b3dd3eb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:20.529937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ca446e78-c301-459d-9685-a1e51b3dd3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.534677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:20.552117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7716cb7-646d-4b73-8a9f-691dae48e740 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.559517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:20.560337 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.560688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.561034 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] PATH_INFO: `/v3/projects/383eaf1769374b1e88af149c0907ad85` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.562322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=383eaf1769374b1e88af149c0907ad85)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:20.564429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:20.604989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1025/2033] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:20.609339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b327a5-26c9-4502-bffb-3b224af577d0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.609629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b327a5-26c9-4502-bffb-3b224af577d0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.609907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b327a5-26c9-4502-bffb-3b224af577d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.616363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383eaf1769374b1e88af149c0907ad85', 'request_id': 'req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.626013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383eaf1769374b1e88af149c0907ad85', 'request_id': 'req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.629623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-25b327a5-26c9-4502-bffb-3b224af577d0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:20.630845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-25b327a5-26c9-4502-bffb-3b224af577d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.633248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383eaf1769374b1e88af149c0907ad85', 'request_id': 'req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.645087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] The token cache is being invalidate because project 383eaf1769374b1e88af149c0907ad85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:20.645378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.646290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:20.649181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1009/2034] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:20 2022] DELETE /identity/v3/projects/383eaf1769374b1e88af149c0907ad85 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:20.655646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.657130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.674365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:20.702626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1026/2035] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:20.704778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6809f2-2c24-4d47-bb86-14f8334f798f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.705204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6809f2-2c24-4d47-bb86-14f8334f798f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.705613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6809f2-2c24-4d47-bb86-14f8334f798f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.731991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5d6809f2-2c24-4d47-bb86-14f8334f798f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:20.733189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5d6809f2-2c24-4d47-bb86-14f8334f798f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.738940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f71bf-ca71-4e65-9f3c-2d5406a902d8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.741135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1027/2036] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:20.742439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394975fc-2f25-49cd-ab88-5cfb77098383 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.742725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394975fc-2f25-49cd-ab88-5cfb77098383 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.742997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394975fc-2f25-49cd-ab88-5cfb77098383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.748415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af499719-574c-4269-af22-d749e7e3b041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:20.749103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af499719-574c-4269-af22-d749e7e3b041 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.749388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af499719-574c-4269-af22-d749e7e3b041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.749672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af499719-574c-4269-af22-d749e7e3b041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.751609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-af499719-574c-4269-af22-d749e7e3b041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.765609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af499719-574c-4269-af22-d749e7e3b041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:20.767216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af499719-574c-4269-af22-d749e7e3b041 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:20.771740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-394975fc-2f25-49cd-ab88-5cfb77098383 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:20.773588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-394975fc-2f25-49cd-ab88-5cfb77098383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.779105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1010/2037] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:20.780447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1421f21-eacc-4a7a-b5be-ee8efdce4dee None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.780725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1421f21-eacc-4a7a-b5be-ee8efdce4dee None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.781032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1421f21-eacc-4a7a-b5be-ee8efdce4dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.783375 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1028/2038] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:20.787542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394975fc-2f25-49cd-ab88-5cfb77098383 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.799149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394975fc-2f25-49cd-ab88-5cfb77098383 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.800632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f1421f21-eacc-4a7a-b5be-ee8efdce4dee None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:20.801827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f1421f21-eacc-4a7a-b5be-ee8efdce4dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:20.812436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:20.813141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.813425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.813701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.814441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:20.814749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:20.816402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:20.819767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1011/2039] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:20.821869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1421f21-eacc-4a7a-b5be-ee8efdce4dee None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.829877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1029/2040] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:20.833998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1421f21-eacc-4a7a-b5be-ee8efdce4dee None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.834159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.839146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:20.839941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.840281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.840631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.841441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:20.841823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:20.843522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:20.845389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3523-d161-4bec-95d9-16b49e3aa131 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.852971 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4addc-e051-4232-888f-9ee976679458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:20.854803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4addc-e051-4232-888f-9ee976679458 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.855092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4addc-e051-4232-888f-9ee976679458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.855396 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4addc-e051-4232-888f-9ee976679458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.856100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4addc-e051-4232-888f-9ee976679458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:20.856656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1012/2041] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:20.861026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.861495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4addc-e051-4232-888f-9ee976679458 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:20.872894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad13758-f2e3-4f31-b549-0a2c5b79b0fa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.879752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb4606-aa99-4595-bb85-d23fbe9b8e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:20.880509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb4606-aa99-4595-bb85-d23fbe9b8e3a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:20.881392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb4606-aa99-4595-bb85-d23fbe9b8e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:20.881392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb4606-aa99-4595-bb85-d23fbe9b8e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:20.882294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb4606-aa99-4595-bb85-d23fbe9b8e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:20.884816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb4606-aa99-4595-bb85-d23fbe9b8e3a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:20.903838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1030/2042] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:20.908153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4addc-e051-4232-888f-9ee976679458 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.923810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1013/2043] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:20.925230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb4606-aa99-4595-bb85-d23fbe9b8e3a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:20.986056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4addc-e051-4232-888f-9ee976679458 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:20.993164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:21.021645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb4606-aa99-4595-bb85-d23fbe9b8e3a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.030728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f757b6f-ea83-45d0-8f0a-95ce58017a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.032080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f757b6f-ea83-45d0-8f0a-95ce58017a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.032819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f757b6f-ea83-45d0-8f0a-95ce58017a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.033383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f757b6f-ea83-45d0-8f0a-95ce58017a99 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.034487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f757b6f-ea83-45d0-8f0a-95ce58017a99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.036929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f757b6f-ea83-45d0-8f0a-95ce58017a99 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.065923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1014/2044] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:21.070543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f757b6f-ea83-45d0-8f0a-95ce58017a99 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.083838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a857f6a-0cb3-47dd-830f-bc78614e06ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.084521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a857f6a-0cb3-47dd-830f-bc78614e06ee admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.084805 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a857f6a-0cb3-47dd-830f-bc78614e06ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.085085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a857f6a-0cb3-47dd-830f-bc78614e06ee admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.085845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a857f6a-0cb3-47dd-830f-bc78614e06ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.089650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a857f6a-0cb3-47dd-830f-bc78614e06ee admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.105893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f757b6f-ea83-45d0-8f0a-95ce58017a99 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.115393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1031/2045] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:20 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:21.120625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a857f6a-0cb3-47dd-830f-bc78614e06ee admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.122042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6855d-2a04-4ae0-945f-fc90b1c85ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.123843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6855d-2a04-4ae0-945f-fc90b1c85ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.124911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6855d-2a04-4ae0-945f-fc90b1c85ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.125505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6855d-2a04-4ae0-945f-fc90b1c85ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.126723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6855d-2a04-4ae0-945f-fc90b1c85ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.129078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6855d-2a04-4ae0-945f-fc90b1c85ad9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.143492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1015/2046] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:21.146177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a857f6a-0cb3-47dd-830f-bc78614e06ee admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.147222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6855d-2a04-4ae0-945f-fc90b1c85ad9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.162935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6855d-2a04-4ae0-945f-fc90b1c85ad9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.168854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1f828-400f-467e-98d7-e5768fe570f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.169594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f828-400f-467e-98d7-e5768fe570f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.170036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f828-400f-467e-98d7-e5768fe570f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.170415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f828-400f-467e-98d7-e5768fe570f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.171886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1f828-400f-467e-98d7-e5768fe570f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.173538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1f828-400f-467e-98d7-e5768fe570f1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.174653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827cc22-a5f5-474c-ab7e-ce15cdb29963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.175581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827cc22-a5f5-474c-ab7e-ce15cdb29963 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.175866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827cc22-a5f5-474c-ab7e-ce15cdb29963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.176632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827cc22-a5f5-474c-ab7e-ce15cdb29963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.178901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827cc22-a5f5-474c-ab7e-ce15cdb29963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.183643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827cc22-a5f5-474c-ab7e-ce15cdb29963 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.189082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1032/2047] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:21.190011 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1f828-400f-467e-98d7-e5768fe570f1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.206154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1f828-400f-467e-98d7-e5768fe570f1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.209429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1016/2048] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:21.211982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827cc22-a5f5-474c-ab7e-ce15cdb29963 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.218078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6128d-862b-4577-95c7-9cf4eda75e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.219102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6128d-862b-4577-95c7-9cf4eda75e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.219843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6128d-862b-4577-95c7-9cf4eda75e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.220321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6128d-862b-4577-95c7-9cf4eda75e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.222001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6128d-862b-4577-95c7-9cf4eda75e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.225802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6128d-862b-4577-95c7-9cf4eda75e20 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.226691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827cc22-a5f5-474c-ab7e-ce15cdb29963 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.240096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1033/2049] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:21.244274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c957a4-5140-4200-9cab-8f518112da6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.244614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6128d-862b-4577-95c7-9cf4eda75e20 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.246425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c957a4-5140-4200-9cab-8f518112da6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.247045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c957a4-5140-4200-9cab-8f518112da6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.247519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c957a4-5140-4200-9cab-8f518112da6d admin admin] PATH_INFO: `/v3/projects/24fc772682c740e69c5b49ce8ec7d8e9/users/69439fcc6e434dde8f47e9314b3a397b/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.258822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6128d-862b-4577-95c7-9cf4eda75e20 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.270695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c957a4-5140-4200-9cab-8f518112da6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fc772682c740e69c5b49ce8ec7d8e9, user_id=69439fcc6e434dde8f47e9314b3a397b, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.274299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c957a4-5140-4200-9cab-8f518112da6d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.281980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dada52-70a8-46ef-8134-b933073bcd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.283447 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dada52-70a8-46ef-8134-b933073bcd2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.284438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dada52-70a8-46ef-8134-b933073bcd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.284856 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dada52-70a8-46ef-8134-b933073bcd2a admin admin] PATH_INFO: `/v3/projects/c2dfa48915864f7dbadea79f5b38e997/users/8c593b130069444a84a8e4d4ecb70f08/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.299407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1017/2050] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:21 2022] PUT /identity/v3/projects/24fc772682c740e69c5b49ce8ec7d8e9/users/69439fcc6e434dde8f47e9314b3a397b/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:21.311676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538be6d-9059-4c28-81a2-66ec8d7a0331 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.312244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538be6d-9059-4c28-81a2-66ec8d7a0331 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.313005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538be6d-9059-4c28-81a2-66ec8d7a0331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.336242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dada52-70a8-46ef-8134-b933073bcd2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2dfa48915864f7dbadea79f5b38e997, user_id=8c593b130069444a84a8e4d4ecb70f08, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.340860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dada52-70a8-46ef-8134-b933073bcd2a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.376334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1034/2051] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:21 2022] PUT /identity/v3/projects/c2dfa48915864f7dbadea79f5b38e997/users/8c593b130069444a84a8e4d4ecb70f08/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:21.378082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e538be6d-9059-4c28-81a2-66ec8d7a0331 None None] MFA Rules not processed for user `69439fcc6e434dde8f47e9314b3a397b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:21.383592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e538be6d-9059-4c28-81a2-66ec8d7a0331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:21.385891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a3795-b9b1-4d73-820b-5f3576b5b7bb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.386178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a3795-b9b1-4d73-820b-5f3576b5b7bb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.386459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a3795-b9b1-4d73-820b-5f3576b5b7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.393906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:21.453106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5e2a3795-b9b1-4d73-820b-5f3576b5b7bb None None] MFA Rules not processed for user `8c593b130069444a84a8e4d4ecb70f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:21.453910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5e2a3795-b9b1-4d73-820b-5f3576b5b7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:21.569482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1018/2052] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:21.569482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538be6d-9059-4c28-81a2-66ec8d7a0331 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.573944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e538be6d-9059-4c28-81a2-66ec8d7a0331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:21.602875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1035/2053] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:21.604517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a3795-b9b1-4d73-820b-5f3576b5b7bb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.605884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5e2a3795-b9b1-4d73-820b-5f3576b5b7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:21.674106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538be6d-9059-4c28-81a2-66ec8d7a0331 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.684535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.685823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.685823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.686155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] PATH_INFO: `/v3/projects/a17f25cc9a2d4fe78900d2aefc296a6c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.687554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17f25cc9a2d4fe78900d2aefc296a6c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.689582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.699435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:21.734395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a3795-b9b1-4d73-820b-5f3576b5b7bb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.755424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17f25cc9a2d4fe78900d2aefc296a6c', 'request_id': 'req-2af34ffc-4091-4ad0-bc68-f4f923346cd3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:21.768953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17f25cc9a2d4fe78900d2aefc296a6c', 'request_id': 'req-2af34ffc-4091-4ad0-bc68-f4f923346cd3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:21.777981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17f25cc9a2d4fe78900d2aefc296a6c', 'request_id': 'req-2af34ffc-4091-4ad0-bc68-f4f923346cd3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:21.795535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] The token cache is being invalidate because project a17f25cc9a2d4fe78900d2aefc296a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:21.796272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:21.797905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:21.801074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1019/2054] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:21 2022] DELETE /identity/v3/projects/a17f25cc9a2d4fe78900d2aefc296a6c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:21.802666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.804316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:21.815325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.821607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.823490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.823490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.827310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:21.863192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:21.899092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af34ffc-4091-4ad0-bc68-f4f923346cd3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:21.908751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:21.909525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:21.909752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:21.910121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:21.916782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:21.933461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:21.934975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:21.946330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1036/2055] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:21.947292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:21.950430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:22.053676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da255b3f-972c-491e-976d-7f6eb2eac640 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:22.063992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:22.064365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:22.064651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:22.065193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] PATH_INFO: `/v3/users/f1910c9c51a84a2f9f7c2c720c22f472` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:22.066456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1910c9c51a84a2f9f7c2c720c22f472)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:22.069832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:22.137623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:22.141515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:22.155399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1020/2056] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:22.184115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1910c9c51a84a2f9f7c2c720c22f472'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:22.184936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1910c9c51a84a2f9f7c2c720c22f472'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:22.193068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:22.200837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1910c9c51a84a2f9f7c2c720c22f472'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:22.221080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1910c9c51a84a2f9f7c2c720c22f472'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:22.222477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-cb7cd723-af7b-49f3-bc95-102862aa7857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1910c9c51a84a2f9f7c2c720c22f472'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:22.242247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1037/2057] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:21 2022] DELETE /identity/v3/users/f1910c9c51a84a2f9f7c2c720c22f472 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:22.928614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:22.959689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9ec04-bcc1-4c91-a61b-3046017a97e3 service glance] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:23.053735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:23.054608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.054895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.055175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] PATH_INFO: `/v3/projects/0f9ac503607545a59356709e9dfa80be` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.058229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9ac503607545a59356709e9dfa80be)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:23.059153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:23.125255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9ac503607545a59356709e9dfa80be', 'request_id': 'req-fb1a3716-f780-4251-9080-13912458646c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:23.146621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9ac503607545a59356709e9dfa80be', 'request_id': 'req-fb1a3716-f780-4251-9080-13912458646c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:23.156022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9ac503607545a59356709e9dfa80be', 'request_id': 'req-fb1a3716-f780-4251-9080-13912458646c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:23.177106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] The token cache is being invalidate because project 0f9ac503607545a59356709e9dfa80be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:23.177410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:23.178090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fb1a3716-f780-4251-9080-13912458646c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:23.181064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1021/2058] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:22 2022] DELETE /identity/v3/projects/0f9ac503607545a59356709e9dfa80be => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:23.186257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d07b-ecaf-4cf1-b53e-4dec1910206b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.186719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d07b-ecaf-4cf1-b53e-4dec1910206b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.186813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d07b-ecaf-4cf1-b53e-4dec1910206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.217458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c6b9d07b-ecaf-4cf1-b53e-4dec1910206b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:23.219356 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c6b9d07b-ecaf-4cf1-b53e-4dec1910206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:23.380248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1038/2059] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:23.388173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c4d96-7d93-4681-96d4-54da31f03937 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.388484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c4d96-7d93-4681-96d4-54da31f03937 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.388746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c4d96-7d93-4681-96d4-54da31f03937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.428522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-704c4d96-7d93-4681-96d4-54da31f03937 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:23.430728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-704c4d96-7d93-4681-96d4-54da31f03937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:23.442298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1022/2060] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:23.446530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9d07b-ecaf-4cf1-b53e-4dec1910206b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:23.464058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9d07b-ecaf-4cf1-b53e-4dec1910206b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:23.474618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:23.475513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.475866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.476407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.480763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:23.481770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:23.483810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:23.515817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1039/2061] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:23.520940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c4d96-7d93-4681-96d4-54da31f03937 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:23.567150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c4d96-7d93-4681-96d4-54da31f03937 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:23.576027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665683e-bff3-4f43-860a-40b0d7bf6fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:23.577759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665683e-bff3-4f43-860a-40b0d7bf6fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.577759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665683e-bff3-4f43-860a-40b0d7bf6fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.577759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665683e-bff3-4f43-860a-40b0d7bf6fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.578545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665683e-bff3-4f43-860a-40b0d7bf6fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:23.580976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665683e-bff3-4f43-860a-40b0d7bf6fc9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:23.595528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:23.672000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1023/2062] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:23 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:23.673054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:23.702575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428e340-1a88-487f-b5d0-b963bf7e6cfd admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:23.775038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444acdd4-3500-4397-a573-ffefb4009d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:23.776606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444acdd4-3500-4397-a573-ffefb4009d83 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.776753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444acdd4-3500-4397-a573-ffefb4009d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.776994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444acdd4-3500-4397-a573-ffefb4009d83 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.777699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444acdd4-3500-4397-a573-ffefb4009d83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:23.779691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444acdd4-3500-4397-a573-ffefb4009d83 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:23.826311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1040/2063] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:23 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:23.830379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665683e-bff3-4f43-860a-40b0d7bf6fc9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:23.842334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665683e-bff3-4f43-860a-40b0d7bf6fc9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:23.849528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a8ae8-e053-47bf-a538-165b60e3fcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:23.851092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a8ae8-e053-47bf-a538-165b60e3fcfc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.851623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a8ae8-e053-47bf-a538-165b60e3fcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.852077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a8ae8-e053-47bf-a538-165b60e3fcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.853330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a8ae8-e053-47bf-a538-165b60e3fcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:23.855269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a8ae8-e053-47bf-a538-165b60e3fcfc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:23.869571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1024/2064] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:23.873570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444acdd4-3500-4397-a573-ffefb4009d83 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:23.884181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444acdd4-3500-4397-a573-ffefb4009d83 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:23.900283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007dd36a-3e22-447f-bca7-f24cb0bad2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:23.901873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007dd36a-3e22-447f-bca7-f24cb0bad2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.902229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007dd36a-3e22-447f-bca7-f24cb0bad2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.903177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007dd36a-3e22-447f-bca7-f24cb0bad2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.904032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007dd36a-3e22-447f-bca7-f24cb0bad2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:23.906163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007dd36a-3e22-447f-bca7-f24cb0bad2a6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:23.919207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1041/2065] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:23.924859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a8ae8-e053-47bf-a538-165b60e3fcfc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:23.939365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a8ae8-e053-47bf-a538-165b60e3fcfc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:23.945770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66cf1-fbeb-45b1-84eb-691cea8389a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:23.946730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66cf1-fbeb-45b1-84eb-691cea8389a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.947711 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66cf1-fbeb-45b1-84eb-691cea8389a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.948244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66cf1-fbeb-45b1-84eb-691cea8389a3 admin admin] PATH_INFO: `/v3/projects/92445246f3084672b988c918f7bc063e/users/2a61b99cd64541ae9861bc1cedd1564e/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:23.967882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66cf1-fbeb-45b1-84eb-691cea8389a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92445246f3084672b988c918f7bc063e, user_id=2a61b99cd64541ae9861bc1cedd1564e, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:23.971487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66cf1-fbeb-45b1-84eb-691cea8389a3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:23.987195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1025/2066] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:23 2022] PUT /identity/v3/projects/92445246f3084672b988c918f7bc063e/users/2a61b99cd64541ae9861bc1cedd1564e/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:23.997758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d0b32-63fe-40b2-9f7d-d7917d555ed5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:23.998151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d0b32-63fe-40b2-9f7d-d7917d555ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:23.998507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d0b32-63fe-40b2-9f7d-d7917d555ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.030366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1b2d0b32-63fe-40b2-9f7d-d7917d555ed5 None None] MFA Rules not processed for user `2a61b99cd64541ae9861bc1cedd1564e`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:24.033034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1b2d0b32-63fe-40b2-9f7d-d7917d555ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:24.074719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:24.145884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1042/2067] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:24.154577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66cf1-fbeb-45b1-84eb-691cea8389a3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.258910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66cf1-fbeb-45b1-84eb-691cea8389a3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.270154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487318-a4f8-45f8-8e11-495750139c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.271092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487318-a4f8-45f8-8e11-495750139c98 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.271591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487318-a4f8-45f8-8e11-495750139c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.272009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487318-a4f8-45f8-8e11-495750139c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.274170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8487318-a4f8-45f8-8e11-495750139c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:24.277793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8487318-a4f8-45f8-8e11-495750139c98 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:24.315162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1026/2068] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:24 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:24.316395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d0b32-63fe-40b2-9f7d-d7917d555ed5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.330289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d0b32-63fe-40b2-9f7d-d7917d555ed5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.391090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d9562-3d05-42c2-b424-a61ac19283d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.391816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9562-3d05-42c2-b424-a61ac19283d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.392098 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9562-3d05-42c2-b424-a61ac19283d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.392374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9562-3d05-42c2-b424-a61ac19283d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.393090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d9562-3d05-42c2-b424-a61ac19283d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:24.395097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d9562-3d05-42c2-b424-a61ac19283d3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:24.422424 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1043/2069] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:24 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:24.424011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487318-a4f8-45f8-8e11-495750139c98 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.434876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487318-a4f8-45f8-8e11-495750139c98 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.440469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f861-0317-4ec1-9028-199c279d91eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.441143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f861-0317-4ec1-9028-199c279d91eb admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.441432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f861-0317-4ec1-9028-199c279d91eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.441748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f861-0317-4ec1-9028-199c279d91eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.442448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037f861-0317-4ec1-9028-199c279d91eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:24.444133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037f861-0317-4ec1-9028-199c279d91eb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:24.457549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1027/2070] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:24.461522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d9562-3d05-42c2-b424-a61ac19283d3 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.474069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d9562-3d05-42c2-b424-a61ac19283d3 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.480794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4f22f-51ec-4e12-a72a-0cbc1dae498f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.481863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4f22f-51ec-4e12-a72a-0cbc1dae498f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.482278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4f22f-51ec-4e12-a72a-0cbc1dae498f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.482892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4f22f-51ec-4e12-a72a-0cbc1dae498f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.483624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4f22f-51ec-4e12-a72a-0cbc1dae498f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:24.485463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4f22f-51ec-4e12-a72a-0cbc1dae498f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:24.503385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1044/2071] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:24.507587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f861-0317-4ec1-9028-199c279d91eb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.518427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f861-0317-4ec1-9028-199c279d91eb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.523260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2fb62-1bdd-40a0-bbdf-bdbc08cce903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.524140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2fb62-1bdd-40a0-bbdf-bdbc08cce903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.524980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2fb62-1bdd-40a0-bbdf-bdbc08cce903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.524980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2fb62-1bdd-40a0-bbdf-bdbc08cce903 admin admin] PATH_INFO: `/v3/projects/904392fbeff142f0bd4b98198e203bd5/users/22cdfadc8cd745a5a0605a17d4b6bbe3/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.552255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2fb62-1bdd-40a0-bbdf-bdbc08cce903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=904392fbeff142f0bd4b98198e203bd5, user_id=22cdfadc8cd745a5a0605a17d4b6bbe3, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:24.554738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2fb62-1bdd-40a0-bbdf-bdbc08cce903 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:24.582840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1028/2072] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:24 2022] PUT /identity/v3/projects/904392fbeff142f0bd4b98198e203bd5/users/22cdfadc8cd745a5a0605a17d4b6bbe3/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:24.593588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cadaf-f776-470a-b949-92b6b52424b7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.593923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cadaf-f776-470a-b949-92b6b52424b7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.594207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cadaf-f776-470a-b949-92b6b52424b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.627722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-aa9cadaf-f776-470a-b949-92b6b52424b7 None None] MFA Rules not processed for user `22cdfadc8cd745a5a0605a17d4b6bbe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:24.629022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-aa9cadaf-f776-470a-b949-92b6b52424b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:24.695168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1045/2073] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:24.700598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2fb62-1bdd-40a0-bbdf-bdbc08cce903 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.774911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2fb62-1bdd-40a0-bbdf-bdbc08cce903 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.782874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692126-f872-49c6-8fff-7a5b0fff2555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.784246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692126-f872-49c6-8fff-7a5b0fff2555 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.784552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692126-f872-49c6-8fff-7a5b0fff2555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.784841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692126-f872-49c6-8fff-7a5b0fff2555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.785529 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc692126-f872-49c6-8fff-7a5b0fff2555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:24.787684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc692126-f872-49c6-8fff-7a5b0fff2555 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:24.798395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:24.818501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1029/2074] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:24 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:24.819846 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cadaf-f776-470a-b949-92b6b52424b7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.833147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cadaf-f776-470a-b949-92b6b52424b7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.870830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:24.899174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826b46b-3a24-4090-9ad3-c9aec1dbda56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.899898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826b46b-3a24-4090-9ad3-c9aec1dbda56 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.901074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826b46b-3a24-4090-9ad3-c9aec1dbda56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.901411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826b46b-3a24-4090-9ad3-c9aec1dbda56 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.902274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826b46b-3a24-4090-9ad3-c9aec1dbda56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:24.904527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826b46b-3a24-4090-9ad3-c9aec1dbda56 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:24.932408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1046/2075] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:24 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:24.936025 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692126-f872-49c6-8fff-7a5b0fff2555 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.948955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692126-f872-49c6-8fff-7a5b0fff2555 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.954241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82683b0a-9dfa-4e00-8343-9da51018eab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.954797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82683b0a-9dfa-4e00-8343-9da51018eab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.955082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82683b0a-9dfa-4e00-8343-9da51018eab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.956253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82683b0a-9dfa-4e00-8343-9da51018eab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:24.957006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82683b0a-9dfa-4e00-8343-9da51018eab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:24.958901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82683b0a-9dfa-4e00-8343-9da51018eab2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:24.972823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1030/2076] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:24.977200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826b46b-3a24-4090-9ad3-c9aec1dbda56 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:24.991206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826b46b-3a24-4090-9ad3-c9aec1dbda56 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:24.996987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3b329-2bfa-43d7-a3f6-237dad9407fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:24.997893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3b329-2bfa-43d7-a3f6-237dad9407fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:24.998196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3b329-2bfa-43d7-a3f6-237dad9407fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:24.998450 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3b329-2bfa-43d7-a3f6-237dad9407fe admin admin] PATH_INFO: `/v3/projects/0da0752896a4401db5e6ea8e02bb68aa/users/70043d8b8dc1423bbff61d24a9a7de5e/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:25.016753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3b329-2bfa-43d7-a3f6-237dad9407fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da0752896a4401db5e6ea8e02bb68aa, user_id=70043d8b8dc1423bbff61d24a9a7de5e, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:25.019460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3b329-2bfa-43d7-a3f6-237dad9407fe admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:25.034992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1047/2077] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:24 2022] PUT /identity/v3/projects/0da0752896a4401db5e6ea8e02bb68aa/users/70043d8b8dc1423bbff61d24a9a7de5e/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:25.039153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82683b0a-9dfa-4e00-8343-9da51018eab2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:25.051732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82683b0a-9dfa-4e00-8343-9da51018eab2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:25.120860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf62f1b-c80e-4749-a982-622004209042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:25.121557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62f1b-c80e-4749-a982-622004209042 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:25.121844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62f1b-c80e-4749-a982-622004209042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:25.122260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62f1b-c80e-4749-a982-622004209042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:25.123289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf62f1b-c80e-4749-a982-622004209042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:25.125502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf62f1b-c80e-4749-a982-622004209042 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:25.143525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1031/2078] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:25.146185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3b329-2bfa-43d7-a3f6-237dad9407fe admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:25.160782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3b329-2bfa-43d7-a3f6-237dad9407fe admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:25.169772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:25.171009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:25.171550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:25.171828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] PATH_INFO: `/v3/projects/0da0752896a4401db5e6ea8e02bb68aa/users/70043d8b8dc1423bbff61d24a9a7de5e/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:25.174003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da0752896a4401db5e6ea8e02bb68aa, user_id=70043d8b8dc1423bbff61d24a9a7de5e, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:25.176769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:25.189262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1048/2079] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:25 2022] PUT /identity/v3/projects/0da0752896a4401db5e6ea8e02bb68aa/users/70043d8b8dc1423bbff61d24a9a7de5e/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:25.200918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b9f39-f1d5-4997-8a0f-c6ec9404a61a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:25.201909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b9f39-f1d5-4997-8a0f-c6ec9404a61a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:25.202241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b9f39-f1d5-4997-8a0f-c6ec9404a61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:25.234539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f72b9f39-f1d5-4997-8a0f-c6ec9404a61a None None] MFA Rules not processed for user `70043d8b8dc1423bbff61d24a9a7de5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:25.235785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f72b9f39-f1d5-4997-8a0f-c6ec9404a61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:25.323806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1032/2080] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:25.332526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:25.334283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:25.405870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474280b9-de5d-418a-8cfd-de3d4ca9b534 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:25.409587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43800630-f14b-4c96-a969-275a361120d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:25.410468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43800630-f14b-4c96-a969-275a361120d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:25.410814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43800630-f14b-4c96-a969-275a361120d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:25.411812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43800630-f14b-4c96-a969-275a361120d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:25.421272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43800630-f14b-4c96-a969-275a361120d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:25.422806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43800630-f14b-4c96-a969-275a361120d3 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:25.436241 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1049/2081] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:25.483117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b9f39-f1d5-4997-8a0f-c6ec9404a61a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:25.484553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f72b9f39-f1d5-4997-8a0f-c6ec9404a61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:25.498643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b9f39-f1d5-4997-8a0f-c6ec9404a61a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:25.502846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb0329-70b4-4fb0-952f-e284a61829d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:25.503609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb0329-70b4-4fb0-952f-e284a61829d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:25.503890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb0329-70b4-4fb0-952f-e284a61829d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:25.504250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb0329-70b4-4fb0-952f-e284a61829d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:25.577668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb0329-70b4-4fb0-952f-e284a61829d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:25.579123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb0329-70b4-4fb0-952f-e284a61829d3 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:25.584258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:25.594281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1033/2082] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:35.432543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=54,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:44.357855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43800630-f14b-4c96-a969-275a361120d3 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:44.424593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43800630-f14b-4c96-a969-275a361120d3 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:44.429743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:44.430750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:44.431110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:44.431487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] PATH_INFO: `/v3/users/2a61b99cd64541ae9861bc1cedd1564e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:44.432802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a61b99cd64541ae9861bc1cedd1564e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:44.434808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:44.487592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a61b99cd64541ae9861bc1cedd1564e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:44.488365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a61b99cd64541ae9861bc1cedd1564e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:44.500132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a61b99cd64541ae9861bc1cedd1564e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:44.513168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a61b99cd64541ae9861bc1cedd1564e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:44.513914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a61b99cd64541ae9861bc1cedd1564e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:44.528413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1050/2083] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:44 2022] DELETE /identity/v3/users/2a61b99cd64541ae9861bc1cedd1564e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:44.533399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4898-968c-4798-87e9-19e915db5067 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:44.533943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4898-968c-4798-87e9-19e915db5067 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:44.534043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4898-968c-4798-87e9-19e915db5067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:44.552761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-17ab4898-968c-4798-87e9-19e915db5067 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:44.553949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-17ab4898-968c-4798-87e9-19e915db5067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:44.613280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1034/2084] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:44.624026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:44.625436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:44.684286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45500ba3-740b-43e0-81a7-34ec6663c00c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:44.687995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a810a2-cef6-4b73-9ed5-cbc9df7df66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:44.688642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a810a2-cef6-4b73-9ed5-cbc9df7df66f service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:44.688927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a810a2-cef6-4b73-9ed5-cbc9df7df66f service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:44.689200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a810a2-cef6-4b73-9ed5-cbc9df7df66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:44.698195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a810a2-cef6-4b73-9ed5-cbc9df7df66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:44.700294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a810a2-cef6-4b73-9ed5-cbc9df7df66f service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:44.710742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1051/2085] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:44.981369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab4898-968c-4798-87e9-19e915db5067 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:44.982966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-17ab4898-968c-4798-87e9-19e915db5067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:44.994960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab4898-968c-4798-87e9-19e915db5067 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:45.000778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:45.001676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:45.001999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:45.002274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] PATH_INFO: `/v3/projects/92445246f3084672b988c918f7bc063e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:45.003625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92445246f3084672b988c918f7bc063e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:45.005433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:45.058305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92445246f3084672b988c918f7bc063e', 'request_id': 'req-443cb42e-8920-4e47-9167-fd0baf9dc427'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.065206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92445246f3084672b988c918f7bc063e', 'request_id': 'req-443cb42e-8920-4e47-9167-fd0baf9dc427'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.071260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92445246f3084672b988c918f7bc063e', 'request_id': 'req-443cb42e-8920-4e47-9167-fd0baf9dc427'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.082000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] The token cache is being invalidate because project 92445246f3084672b988c918f7bc063e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:45.082267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.082889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.084453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1035/2086] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:44 2022] DELETE /identity/v3/projects/92445246f3084672b988c918f7bc063e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:45.088399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a810a2-cef6-4b73-9ed5-cbc9df7df66f service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:45.097918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a810a2-cef6-4b73-9ed5-cbc9df7df66f service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:45.115575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:45.157598 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:45.158407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:45.158691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:45.158976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] PATH_INFO: `/v3/users/22cdfadc8cd745a5a0605a17d4b6bbe3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:45.160229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22cdfadc8cd745a5a0605a17d4b6bbe3)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:45.162257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:45.220025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22cdfadc8cd745a5a0605a17d4b6bbe3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.220698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22cdfadc8cd745a5a0605a17d4b6bbe3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.227520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22cdfadc8cd745a5a0605a17d4b6bbe3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.239201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22cdfadc8cd745a5a0605a17d4b6bbe3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.239866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22cdfadc8cd745a5a0605a17d4b6bbe3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.253462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1052/2087] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:45 2022] DELETE /identity/v3/users/22cdfadc8cd745a5a0605a17d4b6bbe3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:45.789138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:45.790605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:45.804349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cb42e-8920-4e47-9167-fd0baf9dc427 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:45.887953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:45.888815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:45.889097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:45.889370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] PATH_INFO: `/v3/projects/904392fbeff142f0bd4b98198e203bd5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:45.890541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=904392fbeff142f0bd4b98198e203bd5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:45.892468 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:45.922822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:45.940308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904392fbeff142f0bd4b98198e203bd5', 'request_id': 'req-6de62e38-37d8-4e3b-8fa0-def3eace1544'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.947512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904392fbeff142f0bd4b98198e203bd5', 'request_id': 'req-6de62e38-37d8-4e3b-8fa0-def3eace1544'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.954426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904392fbeff142f0bd4b98198e203bd5', 'request_id': 'req-6de62e38-37d8-4e3b-8fa0-def3eace1544'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.966206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] The token cache is being invalidate because project 904392fbeff142f0bd4b98198e203bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:45.966507 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.967147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:45.968723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1036/2088] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:45 2022] DELETE /identity/v3/projects/904392fbeff142f0bd4b98198e203bd5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:45.973095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:45.975153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:45.988495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb591fe-b7de-4e0b-9f67-6b39305b14a2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:46.046167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:46.047098 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.047401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.047678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] PATH_INFO: `/v3/users/70043d8b8dc1423bbff61d24a9a7de5e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.048943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70043d8b8dc1423bbff61d24a9a7de5e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:46.050891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:46.097095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:46.100043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70043d8b8dc1423bbff61d24a9a7de5e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.101295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70043d8b8dc1423bbff61d24a9a7de5e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.109303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70043d8b8dc1423bbff61d24a9a7de5e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.122273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70043d8b8dc1423bbff61d24a9a7de5e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.122942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8c6523f6-8566-43e8-b28e-2a7b0a6479b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70043d8b8dc1423bbff61d24a9a7de5e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.135212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1053/2089] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:45 2022] DELETE /identity/v3/users/70043d8b8dc1423bbff61d24a9a7de5e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:46.412465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:46.413913 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:46.431823 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de62e38-37d8-4e3b-8fa0-def3eace1544 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:46.508036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:46.508916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.509209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.509492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] PATH_INFO: `/v3/projects/0da0752896a4401db5e6ea8e02bb68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.510696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da0752896a4401db5e6ea8e02bb68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:46.512632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:46.565514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da0752896a4401db5e6ea8e02bb68aa', 'request_id': 'req-3a07167a-8741-4726-87a3-8225331cf576'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.576037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da0752896a4401db5e6ea8e02bb68aa', 'request_id': 'req-3a07167a-8741-4726-87a3-8225331cf576'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.583038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da0752896a4401db5e6ea8e02bb68aa', 'request_id': 'req-3a07167a-8741-4726-87a3-8225331cf576'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.594406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] The token cache is being invalidate because project 0da0752896a4401db5e6ea8e02bb68aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:46.594695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.595373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a07167a-8741-4726-87a3-8225331cf576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:46.596950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1037/2090] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:46 2022] DELETE /identity/v3/projects/0da0752896a4401db5e6ea8e02bb68aa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:46.603166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296e998-9edf-4bff-a720-18743ffbc37a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.603519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296e998-9edf-4bff-a720-18743ffbc37a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.603802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296e998-9edf-4bff-a720-18743ffbc37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.625023 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7296e998-9edf-4bff-a720-18743ffbc37a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:46.626294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7296e998-9edf-4bff-a720-18743ffbc37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:46.691135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1054/2091] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:46.699606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b496fc-9bf1-417c-8e18-d75f916af213 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.700145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b496fc-9bf1-417c-8e18-d75f916af213 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.700422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b496fc-9bf1-417c-8e18-d75f916af213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.720072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b7b496fc-9bf1-417c-8e18-d75f916af213 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:46.721315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b7b496fc-9bf1-417c-8e18-d75f916af213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:46.733239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1038/2092] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:46.734705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296e998-9edf-4bff-a720-18743ffbc37a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:46.744988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296e998-9edf-4bff-a720-18743ffbc37a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:46.750600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:46.751265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.751586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.751862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.752603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:46.752910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:46.754524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:46.770488 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1055/2093] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:26:46.771900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b496fc-9bf1-417c-8e18-d75f916af213 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:46.782219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b496fc-9bf1-417c-8e18-d75f916af213 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:46.786944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393105e6-8c00-4e78-9002-d54d7a86121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:46.787643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393105e6-8c00-4e78-9002-d54d7a86121a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.787934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393105e6-8c00-4e78-9002-d54d7a86121a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.788209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393105e6-8c00-4e78-9002-d54d7a86121a admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.788952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393105e6-8c00-4e78-9002-d54d7a86121a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:46.790868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393105e6-8c00-4e78-9002-d54d7a86121a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:46.820310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1039/2094] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:46 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:46.821525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:46.832717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e8ba-a42e-4928-9ba2-5b6385a7fbdb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:46.900457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb1daf-2133-4a37-b9ef-19090bb04f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:46.901152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1daf-2133-4a37-b9ef-19090bb04f02 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.901565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1daf-2133-4a37-b9ef-19090bb04f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.901851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1daf-2133-4a37-b9ef-19090bb04f02 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.902579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb1daf-2133-4a37-b9ef-19090bb04f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:46.904600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb1daf-2133-4a37-b9ef-19090bb04f02 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:46.929250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1056/2095] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:46 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:46.930435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393105e6-8c00-4e78-9002-d54d7a86121a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:46.941492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393105e6-8c00-4e78-9002-d54d7a86121a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:46.946138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379cedd-d9d2-4046-ad76-7f9ce397ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:46.946837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379cedd-d9d2-4046-ad76-7f9ce397ee61 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.947120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379cedd-d9d2-4046-ad76-7f9ce397ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.947590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379cedd-d9d2-4046-ad76-7f9ce397ee61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.948165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379cedd-d9d2-4046-ad76-7f9ce397ee61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:46.950459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379cedd-d9d2-4046-ad76-7f9ce397ee61 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:46.963483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1040/2096] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:46.967874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb1daf-2133-4a37-b9ef-19090bb04f02 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:46.977968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb1daf-2133-4a37-b9ef-19090bb04f02 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:46.983649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e27f21-5a3d-4ac5-9d33-ba5557d1e17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:46.984309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e27f21-5a3d-4ac5-9d33-ba5557d1e17d admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:46.984589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e27f21-5a3d-4ac5-9d33-ba5557d1e17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:46.984862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e27f21-5a3d-4ac5-9d33-ba5557d1e17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:46.985602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e27f21-5a3d-4ac5-9d33-ba5557d1e17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:46.987230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e27f21-5a3d-4ac5-9d33-ba5557d1e17d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:46.998784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1057/2097] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:47.002811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379cedd-d9d2-4046-ad76-7f9ce397ee61 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:47.010554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:47.015106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379cedd-d9d2-4046-ad76-7f9ce397ee61 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:47.021049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc27dd-b40b-41e9-802f-2c954076f7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:47.022109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc27dd-b40b-41e9-802f-2c954076f7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.022611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc27dd-b40b-41e9-802f-2c954076f7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.022942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc27dd-b40b-41e9-802f-2c954076f7c9 admin admin] PATH_INFO: `/v3/projects/92bb84085dc44edc9b7c7467d6735ae9/users/42c61eb54b874374b6528d20b2ac2332/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.038040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc27dd-b40b-41e9-802f-2c954076f7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92bb84085dc44edc9b7c7467d6735ae9, user_id=42c61eb54b874374b6528d20b2ac2332, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:47.040706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc27dd-b40b-41e9-802f-2c954076f7c9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:47.053402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1041/2098] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:47 2022] PUT /identity/v3/projects/92bb84085dc44edc9b7c7467d6735ae9/users/42c61eb54b874374b6528d20b2ac2332/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:47.065223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae6602-e485-4c4e-bf4f-82acb50b67d6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.065508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae6602-e485-4c4e-bf4f-82acb50b67d6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.065784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae6602-e485-4c4e-bf4f-82acb50b67d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.092957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-12ae6602-e485-4c4e-bf4f-82acb50b67d6 None None] MFA Rules not processed for user `42c61eb54b874374b6528d20b2ac2332`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:47.094185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-12ae6602-e485-4c4e-bf4f-82acb50b67d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:47.107469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:47.161368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1058/2099] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:47.165994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc27dd-b40b-41e9-802f-2c954076f7c9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:47.227314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc27dd-b40b-41e9-802f-2c954076f7c9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:47.232483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b2b1-4adb-4abd-9cdf-f2380fe67ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:47.233138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b2b1-4adb-4abd-9cdf-f2380fe67ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.233422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b2b1-4adb-4abd-9cdf-f2380fe67ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.233698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b2b1-4adb-4abd-9cdf-f2380fe67ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.234426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73b2b1-4adb-4abd-9cdf-f2380fe67ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:47.236348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73b2b1-4adb-4abd-9cdf-f2380fe67ad1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:47.261103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1042/2100] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:47 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:47.268871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae6602-e485-4c4e-bf4f-82acb50b67d6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:47.278101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae6602-e485-4c4e-bf4f-82acb50b67d6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:47.335752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b8e24-06e1-4142-913d-5e397b9e5005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:47.336415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b8e24-06e1-4142-913d-5e397b9e5005 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.336693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b8e24-06e1-4142-913d-5e397b9e5005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.336991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b8e24-06e1-4142-913d-5e397b9e5005 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.337681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b8e24-06e1-4142-913d-5e397b9e5005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:47.340218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b8e24-06e1-4142-913d-5e397b9e5005 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:47.360316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1059/2101] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:47 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:47.364241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b2b1-4adb-4abd-9cdf-f2380fe67ad1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:47.375361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b2b1-4adb-4abd-9cdf-f2380fe67ad1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:47.380167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe5b15-51d7-43e9-86e1-0f1aa95a5dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:47.380865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5b15-51d7-43e9-86e1-0f1aa95a5dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.381149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5b15-51d7-43e9-86e1-0f1aa95a5dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.381454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5b15-51d7-43e9-86e1-0f1aa95a5dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.382272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5b15-51d7-43e9-86e1-0f1aa95a5dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:47.383897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5b15-51d7-43e9-86e1-0f1aa95a5dcd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:47.394861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1043/2102] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:47.399065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b8e24-06e1-4142-913d-5e397b9e5005 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:47.410468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b8e24-06e1-4142-913d-5e397b9e5005 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:47.415512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0a51c-ac04-4752-bb13-b1a7fc88601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:47.416388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0a51c-ac04-4752-bb13-b1a7fc88601f admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.416668 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0a51c-ac04-4752-bb13-b1a7fc88601f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.416939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0a51c-ac04-4752-bb13-b1a7fc88601f admin admin] PATH_INFO: `/v3/projects/17d8a98526f64152bd8ff4c367885bf1/users/42368deb5e8b4fc2a2e567de3fa3c2f6/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.433588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0a51c-ac04-4752-bb13-b1a7fc88601f admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d8a98526f64152bd8ff4c367885bf1, user_id=42368deb5e8b4fc2a2e567de3fa3c2f6, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:47.436276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0a51c-ac04-4752-bb13-b1a7fc88601f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:47.448525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1060/2103] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:47 2022] PUT /identity/v3/projects/17d8a98526f64152bd8ff4c367885bf1/users/42368deb5e8b4fc2a2e567de3fa3c2f6/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:47.452484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe5b15-51d7-43e9-86e1-0f1aa95a5dcd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:47.462301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe5b15-51d7-43e9-86e1-0f1aa95a5dcd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:47.526116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c65dd-ed09-4334-a322-1990f888dae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:47.527579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c65dd-ed09-4334-a322-1990f888dae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.527903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c65dd-ed09-4334-a322-1990f888dae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.528142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c65dd-ed09-4334-a322-1990f888dae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.528874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c65dd-ed09-4334-a322-1990f888dae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:47.530645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c65dd-ed09-4334-a322-1990f888dae2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:47.542706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1044/2104] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:47.545156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0a51c-ac04-4752-bb13-b1a7fc88601f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:47.555246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0a51c-ac04-4752-bb13-b1a7fc88601f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:47.560038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:47.560911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.561187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.561456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] PATH_INFO: `/v3/projects/17d8a98526f64152bd8ff4c367885bf1/users/42368deb5e8b4fc2a2e567de3fa3c2f6/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.563438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d8a98526f64152bd8ff4c367885bf1, user_id=42368deb5e8b4fc2a2e567de3fa3c2f6, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:47.566362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:47.576652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1061/2105] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:47 2022] PUT /identity/v3/projects/17d8a98526f64152bd8ff4c367885bf1/users/42368deb5e8b4fc2a2e567de3fa3c2f6/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:47.587254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62684515-27ac-414f-9779-49690be1decf None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.587572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62684515-27ac-414f-9779-49690be1decf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.587877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62684515-27ac-414f-9779-49690be1decf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.614758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-62684515-27ac-414f-9779-49690be1decf None None] MFA Rules not processed for user `42368deb5e8b4fc2a2e567de3fa3c2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:47.615991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-62684515-27ac-414f-9779-49690be1decf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:47.656432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:47.685834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1045/2106] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:26:47.696676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:47.698472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:47.759192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee2971-ffbb-4559-8c11-b830c182bf1f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:47.763366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411060cf-55c7-4459-99fa-05e363ed8dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:47.764077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411060cf-55c7-4459-99fa-05e363ed8dbd service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:47.764300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411060cf-55c7-4459-99fa-05e363ed8dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:47.764574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411060cf-55c7-4459-99fa-05e363ed8dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:47.773457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411060cf-55c7-4459-99fa-05e363ed8dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:47.774878 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411060cf-55c7-4459-99fa-05e363ed8dbd service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:47.785024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1062/2107] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:48.183382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62684515-27ac-414f-9779-49690be1decf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:48.244010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62684515-27ac-414f-9779-49690be1decf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:48.248857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:48.249732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:48.250012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:48.250289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] PATH_INFO: `/v3/users/42c61eb54b874374b6528d20b2ac2332` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:48.251510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c61eb54b874374b6528d20b2ac2332)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:48.254408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:48.300195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c61eb54b874374b6528d20b2ac2332'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.307794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c61eb54b874374b6528d20b2ac2332'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.319505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c61eb54b874374b6528d20b2ac2332'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.321857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c61eb54b874374b6528d20b2ac2332'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.329434 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c61eb54b874374b6528d20b2ac2332'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.335868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1046/2108] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:48 2022] DELETE /identity/v3/users/42c61eb54b874374b6528d20b2ac2332 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:48.341635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9b08f-2cbe-476d-b98d-e8eff534086f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:48.342106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9b08f-2cbe-476d-b98d-e8eff534086f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:48.342195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9b08f-2cbe-476d-b98d-e8eff534086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:48.360766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d7d9b08f-2cbe-476d-b98d-e8eff534086f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:48.361976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d7d9b08f-2cbe-476d-b98d-e8eff534086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:48.367189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:48.423974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1063/2109] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:48.433814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:48.435210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:48.494174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c162df2-d541-4ec7-8679-273a91556713 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:48.498204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da95e-ea47-45be-8bf2-dbde74a41129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:48.498855 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da95e-ea47-45be-8bf2-dbde74a41129 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:48.499135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da95e-ea47-45be-8bf2-dbde74a41129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:48.499440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da95e-ea47-45be-8bf2-dbde74a41129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:48.508791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80da95e-ea47-45be-8bf2-dbde74a41129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:48.510228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80da95e-ea47-45be-8bf2-dbde74a41129 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:48.515049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:48.527683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1047/2110] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:26:48.800172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9b08f-2cbe-476d-b98d-e8eff534086f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:48.801605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d7d9b08f-2cbe-476d-b98d-e8eff534086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:48.813514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9b08f-2cbe-476d-b98d-e8eff534086f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:48.818235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:48.819110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:48.819711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:48.820031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] PATH_INFO: `/v3/projects/92bb84085dc44edc9b7c7467d6735ae9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:48.821246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=92bb84085dc44edc9b7c7467d6735ae9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:48.823943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:48.871991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92bb84085dc44edc9b7c7467d6735ae9', 'request_id': 'req-35f2b060-7d3a-43da-a0c9-30582c8eea4e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.878272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92bb84085dc44edc9b7c7467d6735ae9', 'request_id': 'req-35f2b060-7d3a-43da-a0c9-30582c8eea4e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.883804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92bb84085dc44edc9b7c7467d6735ae9', 'request_id': 'req-35f2b060-7d3a-43da-a0c9-30582c8eea4e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.897090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] The token cache is being invalidate because project 92bb84085dc44edc9b7c7467d6735ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:48.897391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.898012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:48.900078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1064/2111] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:48 2022] DELETE /identity/v3/projects/92bb84085dc44edc9b7c7467d6735ae9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:48.903930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da95e-ea47-45be-8bf2-dbde74a41129 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:48.913692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da95e-ea47-45be-8bf2-dbde74a41129 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:48.972194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:48.973075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:48.973355 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:48.973630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] PATH_INFO: `/v3/users/42368deb5e8b4fc2a2e567de3fa3c2f6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:48.974812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42368deb5e8b4fc2a2e567de3fa3c2f6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:48.976770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:49.023189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42368deb5e8b4fc2a2e567de3fa3c2f6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.031803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42368deb5e8b4fc2a2e567de3fa3c2f6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.043800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42368deb5e8b4fc2a2e567de3fa3c2f6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.044465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42368deb5e8b4fc2a2e567de3fa3c2f6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.051174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0315cea-72f0-45b0-9532-4f5d423fb43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42368deb5e8b4fc2a2e567de3fa3c2f6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.057205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1048/2112] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:26:48 2022] DELETE /identity/v3/users/42368deb5e8b4fc2a2e567de3fa3c2f6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:49.494814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:49.497157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:49.510132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2b060-7d3a-43da-a0c9-30582c8eea4e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:49.577989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:49.578864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:49.579155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:49.579462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] PATH_INFO: `/v3/projects/17d8a98526f64152bd8ff4c367885bf1` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:49.580649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d8a98526f64152bd8ff4c367885bf1)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:49.582567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:49.624046 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:49.634836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d8a98526f64152bd8ff4c367885bf1', 'request_id': 'req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.641367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d8a98526f64152bd8ff4c367885bf1', 'request_id': 'req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.647169 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d8a98526f64152bd8ff4c367885bf1', 'request_id': 'req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.659969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] The token cache is being invalidate because project 17d8a98526f64152bd8ff4c367885bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:26:49.660255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.660869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-531a4e25-9339-46a2-91b4-c8f7e4f1a7dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:26:49.662404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1065/2113] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:26:49 2022] DELETE /identity/v3/projects/17d8a98526f64152bd8ff4c367885bf1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:49.668113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f7ebb-1a18-4a43-a89c-90c32ca74ac3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:49.668627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f7ebb-1a18-4a43-a89c-90c32ca74ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:49.668822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f7ebb-1a18-4a43-a89c-90c32ca74ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:49.687814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6b0f7ebb-1a18-4a43-a89c-90c32ca74ac3 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:49.689041 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6b0f7ebb-1a18-4a43-a89c-90c32ca74ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:49.749533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1049/2114] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:49.757615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6940377-7edf-4896-a815-5a4de4db30dd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:49.757908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6940377-7edf-4896-a815-5a4de4db30dd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:49.758513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6940377-7edf-4896-a815-5a4de4db30dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:49.777576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a6940377-7edf-4896-a815-5a4de4db30dd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:49.778807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a6940377-7edf-4896-a815-5a4de4db30dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:49.787203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1066/2115] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:49.791384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f7ebb-1a18-4a43-a89c-90c32ca74ac3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:49.801550 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f7ebb-1a18-4a43-a89c-90c32ca74ac3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:49.806314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:49.806999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:49.807326 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:49.807761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:49.808564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:26:49.808904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:49.810611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:49.822269 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1050/2116] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:26:49.827332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6940377-7edf-4896-a815-5a4de4db30dd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:49.836858 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6940377-7edf-4896-a815-5a4de4db30dd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:49.842187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a34a02-6232-4352-885c-8d6248da2adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:49.842839 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34a02-6232-4352-885c-8d6248da2adb admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:49.843117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34a02-6232-4352-885c-8d6248da2adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:49.843420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34a02-6232-4352-885c-8d6248da2adb admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:49.844120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a34a02-6232-4352-885c-8d6248da2adb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:49.846018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a34a02-6232-4352-885c-8d6248da2adb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:49.871852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1067/2117] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:49 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:49.875857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:49.882083 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:49.886000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fada6-f2b1-42f1-ab34-2037f13dbc93 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:49.943162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56b08a-22c9-4838-aee1-7c455869478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:49.943861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56b08a-22c9-4838-aee1-7c455869478f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:49.944146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56b08a-22c9-4838-aee1-7c455869478f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:49.944432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56b08a-22c9-4838-aee1-7c455869478f admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:49.945157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56b08a-22c9-4838-aee1-7c455869478f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:49.947062 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56b08a-22c9-4838-aee1-7c455869478f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:49.968231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1051/2118] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:49 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:49.972329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a34a02-6232-4352-885c-8d6248da2adb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:49.983087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a34a02-6232-4352-885c-8d6248da2adb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:49.987910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c172c3-c25a-4b56-b1b7-5e267374ddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:49.988574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c172c3-c25a-4b56-b1b7-5e267374ddaa admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:49.988862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c172c3-c25a-4b56-b1b7-5e267374ddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:49.989150 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c172c3-c25a-4b56-b1b7-5e267374ddaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:49.989887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c172c3-c25a-4b56-b1b7-5e267374ddaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:49.991535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c172c3-c25a-4b56-b1b7-5e267374ddaa admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:50.002295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1068/2119] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:26:50.006258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56b08a-22c9-4838-aee1-7c455869478f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:50.017253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56b08a-22c9-4838-aee1-7c455869478f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:50.022010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43ccf1-a00e-4694-b895-fa8f34c54975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:50.022746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43ccf1-a00e-4694-b895-fa8f34c54975 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:50.023028 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43ccf1-a00e-4694-b895-fa8f34c54975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:50.023307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43ccf1-a00e-4694-b895-fa8f34c54975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:50.024065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43ccf1-a00e-4694-b895-fa8f34c54975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:50.025660 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43ccf1-a00e-4694-b895-fa8f34c54975 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:50.038382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1052/2120] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:50.042955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c172c3-c25a-4b56-b1b7-5e267374ddaa admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:50.053329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c172c3-c25a-4b56-b1b7-5e267374ddaa admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:50.058008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:50.059019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:50.059298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:50.059606 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] PATH_INFO: `/v3/projects/3cba1f063ee7455c8419c1dceba4ac45/users/c634b1b55ee94c4d82990d4e37359b62/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:50.074353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cba1f063ee7455c8419c1dceba4ac45, user_id=c634b1b55ee94c4d82990d4e37359b62, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:50.077128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:50.090353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1069/2121] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:50 2022] PUT /identity/v3/projects/3cba1f063ee7455c8419c1dceba4ac45/users/c634b1b55ee94c4d82990d4e37359b62/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:26:50.095363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d56b1-5638-4c9d-9b30-d57ce56d63ff None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:50.095657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d56b1-5638-4c9d-9b30-d57ce56d63ff None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:50.095940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d56b1-5638-4c9d-9b30-d57ce56d63ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:50.113609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6e9d56b1-5638-4c9d-9b30-d57ce56d63ff None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:50.114807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e9d56b1-5638-4c9d-9b30-d57ce56d63ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:50.172921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1053/2122] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:50.182478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:50.184226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:50.241569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7d68b-d423-4e36-8ff2-8e7299b99614 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:50.246695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebac51e-2d49-4932-8304-ae261d97dc09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:50.247369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebac51e-2d49-4932-8304-ae261d97dc09 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:50.247703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebac51e-2d49-4932-8304-ae261d97dc09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:50.247994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebac51e-2d49-4932-8304-ae261d97dc09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:50.280208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebac51e-2d49-4932-8304-ae261d97dc09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:50.281995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebac51e-2d49-4932-8304-ae261d97dc09 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:50.303221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1070/2123] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:51.446216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ebaa4-ae13-42ef-9733-446d18cbb58e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:51.447096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ebaa4-ae13-42ef-9733-446d18cbb58e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:51.447492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ebaa4-ae13-42ef-9733-446d18cbb58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:51.475256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f47ebaa4-ae13-42ef-9733-446d18cbb58e None None] MFA Rules not processed for user `c634b1b55ee94c4d82990d4e37359b62`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:51.476750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f47ebaa4-ae13-42ef-9733-446d18cbb58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:51.562820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebac51e-2d49-4932-8304-ae261d97dc09 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:51.570534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1054/2124] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:51.582903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebac51e-2d49-4932-8304-ae261d97dc09 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:51.588499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401da64b-1579-41f3-8cad-a59a94a88acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:51.589618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da64b-1579-41f3-8cad-a59a94a88acd admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:51.589975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da64b-1579-41f3-8cad-a59a94a88acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:51.590335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da64b-1579-41f3-8cad-a59a94a88acd admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:51.591099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401da64b-1579-41f3-8cad-a59a94a88acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:51.593501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401da64b-1579-41f3-8cad-a59a94a88acd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:51.647533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1071/2125] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:26:51.648191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ebaa4-ae13-42ef-9733-446d18cbb58e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:51.659256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ebaa4-ae13-42ef-9733-446d18cbb58e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:51.668832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:51.728425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b56d55-788e-4ed9-b09a-6e80f05f8643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:51.729156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56d55-788e-4ed9-b09a-6e80f05f8643 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:51.729991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56d55-788e-4ed9-b09a-6e80f05f8643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:51.730458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56d55-788e-4ed9-b09a-6e80f05f8643 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:51.731237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b56d55-788e-4ed9-b09a-6e80f05f8643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:51.737019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b56d55-788e-4ed9-b09a-6e80f05f8643 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:51.766461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1055/2126] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:26:51 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:26:51.770707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401da64b-1579-41f3-8cad-a59a94a88acd admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:51.781723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401da64b-1579-41f3-8cad-a59a94a88acd admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:51.786810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b363ca-182f-4c50-87b2-514769443b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:51.787692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b363ca-182f-4c50-87b2-514769443b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:51.788044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b363ca-182f-4c50-87b2-514769443b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:51.789945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b363ca-182f-4c50-87b2-514769443b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:51.790808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b363ca-182f-4c50-87b2-514769443b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:51.793369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b363ca-182f-4c50-87b2-514769443b48 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:51.805121 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1072/2127] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:51.810066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b56d55-788e-4ed9-b09a-6e80f05f8643 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:51.826197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b56d55-788e-4ed9-b09a-6e80f05f8643 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:51.838003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28a2f2-f1d4-457e-86c6-9ac4ced0957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:51.840104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28a2f2-f1d4-457e-86c6-9ac4ced0957f admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:51.840490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28a2f2-f1d4-457e-86c6-9ac4ced0957f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:51.840852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28a2f2-f1d4-457e-86c6-9ac4ced0957f admin admin] PATH_INFO: `/v3/projects/1debbe36ef274dcda31b1fca14faa365/users/2fc78cd8cfd54a2a96869d3f019ae64d/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:51.866486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28a2f2-f1d4-457e-86c6-9ac4ced0957f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1debbe36ef274dcda31b1fca14faa365, user_id=2fc78cd8cfd54a2a96869d3f019ae64d, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:51.869462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28a2f2-f1d4-457e-86c6-9ac4ced0957f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:51.885361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1056/2128] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:51 2022] PUT /identity/v3/projects/1debbe36ef274dcda31b1fca14faa365/users/2fc78cd8cfd54a2a96869d3f019ae64d/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:51.889281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b363ca-182f-4c50-87b2-514769443b48 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:51.899686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b363ca-182f-4c50-87b2-514769443b48 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:51.951052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:26:51.985422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477181ca-fe4e-49ca-ada0-d4339bb383e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:51.986089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477181ca-fe4e-49ca-ada0-d4339bb383e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:51.986457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477181ca-fe4e-49ca-ada0-d4339bb383e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:51.986755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477181ca-fe4e-49ca-ada0-d4339bb383e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:51.987526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477181ca-fe4e-49ca-ada0-d4339bb383e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:51.989320 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477181ca-fe4e-49ca-ada0-d4339bb383e6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:52.003775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1073/2129] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:26:52.005004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28a2f2-f1d4-457e-86c6-9ac4ced0957f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:52.016457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28a2f2-f1d4-457e-86c6-9ac4ced0957f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:52.021230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:52.022115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:52.022397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:52.022672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] PATH_INFO: `/v3/projects/1debbe36ef274dcda31b1fca14faa365/users/2fc78cd8cfd54a2a96869d3f019ae64d/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:52.024726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1debbe36ef274dcda31b1fca14faa365, user_id=2fc78cd8cfd54a2a96869d3f019ae64d, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:52.028133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:52.038795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1057/2130] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:26:52 2022] PUT /identity/v3/projects/1debbe36ef274dcda31b1fca14faa365/users/2fc78cd8cfd54a2a96869d3f019ae64d/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:26:52.719778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42f75-d3f5-44d5-97d2-75d3e9516168 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:52.720661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42f75-d3f5-44d5-97d2-75d3e9516168 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:52.721087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42f75-d3f5-44d5-97d2-75d3e9516168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:52.804704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-69c42f75-d3f5-44d5-97d2-75d3e9516168 None None] MFA Rules not processed for user `2fc78cd8cfd54a2a96869d3f019ae64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:26:52.806888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-69c42f75-d3f5-44d5-97d2-75d3e9516168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:52.934467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1074/2131] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:26:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:26:52.953546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:52.955699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:53.060049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc4e5d-0d36-484a-a04f-a222f5c8cf97 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:53.063810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:53.064523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:53.065034 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:53.065578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:53.143222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:53.144822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:53.155469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1058/2132] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:26:56.695124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c42f75-d3f5-44d5-97d2-75d3e9516168 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:26:56.696572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-69c42f75-d3f5-44d5-97d2-75d3e9516168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:26:56.709128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c42f75-d3f5-44d5-97d2-75d3e9516168 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:26:56.720615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:26:56.721073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:26:56.721355 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:26:56.721631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:26:56.733481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:26:56.734982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:26:56.745937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1075/2133] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:26:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:03.150044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:06.740463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=55 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:08.953178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:08.954910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:08.966799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:08.966799 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.037374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726f2e5-e349-426f-bf5c-209fee21afa0 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:09.042582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:09.043751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.044129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.044497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] PATH_INFO: `/v3/users/8c593b130069444a84a8e4d4ecb70f08` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.046298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c593b130069444a84a8e4d4ecb70f08)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:09.048361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:09.055980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423ce90-f034-4825-81f8-d84a1e15cd05 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:09.060078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:09.061494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.061976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.062094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] PATH_INFO: `/v3/users/69439fcc6e434dde8f47e9314b3a397b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.063533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69439fcc6e434dde8f47e9314b3a397b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:09.065675 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:09.109017 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c593b130069444a84a8e4d4ecb70f08'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.109829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c593b130069444a84a8e4d4ecb70f08'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.119631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c593b130069444a84a8e4d4ecb70f08'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.131228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69439fcc6e434dde8f47e9314b3a397b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.135648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c593b130069444a84a8e4d4ecb70f08'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.136417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13664998-4f4a-418c-9c21-31197be64984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c593b130069444a84a8e4d4ecb70f08'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.141626 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69439fcc6e434dde8f47e9314b3a397b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.151438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1076/2134] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:08 2022] DELETE /identity/v3/users/8c593b130069444a84a8e4d4ecb70f08 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:09.156007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959bce1-7204-4bf1-87ab-d217d056fc27 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.156356 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959bce1-7204-4bf1-87ab-d217d056fc27 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.157060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959bce1-7204-4bf1-87ab-d217d056fc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.157979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69439fcc6e434dde8f47e9314b3a397b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.158700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69439fcc6e434dde8f47e9314b3a397b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.167056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ef649810-23a6-4edf-955c-3e21a7c4de29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69439fcc6e434dde8f47e9314b3a397b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.174826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1059/2135] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:08 2022] DELETE /identity/v3/users/69439fcc6e434dde8f47e9314b3a397b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:09.176924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f959bce1-7204-4bf1-87ab-d217d056fc27 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:09.178230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f959bce1-7204-4bf1-87ab-d217d056fc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.180012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8b21b-415c-4d12-8953-0db0776b0e49 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.180490 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8b21b-415c-4d12-8953-0db0776b0e49 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.180780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8b21b-415c-4d12-8953-0db0776b0e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.201075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4af8b21b-415c-4d12-8953-0db0776b0e49 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:09.202723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4af8b21b-415c-4d12-8953-0db0776b0e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.248970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1077/2136] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:09.259090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959bce1-7204-4bf1-87ab-d217d056fc27 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:09.260919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f959bce1-7204-4bf1-87ab-d217d056fc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.282793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1060/2137] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:09.289879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:09.301883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8b21b-415c-4d12-8953-0db0776b0e49 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:09.304119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4af8b21b-415c-4d12-8953-0db0776b0e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.328740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:09.338760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959bce1-7204-4bf1-87ab-d217d056fc27 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:09.343435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea8bcd-6cc8-4d40-aa51-2d1afd79fe2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:09.344094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea8bcd-6cc8-4d40-aa51-2d1afd79fe2b service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.344374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea8bcd-6cc8-4d40-aa51-2d1afd79fe2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.344711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea8bcd-6cc8-4d40-aa51-2d1afd79fe2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.354868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea8bcd-6cc8-4d40-aa51-2d1afd79fe2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:09.356542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea8bcd-6cc8-4d40-aa51-2d1afd79fe2b service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:09.367385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1078/2138] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:27:09.372798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8b21b-415c-4d12-8953-0db0776b0e49 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:09.377986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad56fa-2ea3-4b38-82f5-759d8655dd91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:09.378706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad56fa-2ea3-4b38-82f5-759d8655dd91 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.378997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad56fa-2ea3-4b38-82f5-759d8655dd91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.379276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad56fa-2ea3-4b38-82f5-759d8655dd91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.390802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad56fa-2ea3-4b38-82f5-759d8655dd91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:09.392406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad56fa-2ea3-4b38-82f5-759d8655dd91 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:09.403937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1061/2139] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:27:09.455100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be89c8-f48a-4da3-9892-cdfa5c21e428 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.455987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be89c8-f48a-4da3-9892-cdfa5c21e428 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.456502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be89c8-f48a-4da3-9892-cdfa5c21e428 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.459103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1079/2140] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:27:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:27:09.465852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590a9df-6c68-43a0-9c95-ddff22042497 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.466390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590a9df-6c68-43a0-9c95-ddff22042497 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.466747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590a9df-6c68-43a0-9c95-ddff22042497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.491521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8590a9df-6c68-43a0-9c95-ddff22042497 None None] MFA Rules not processed for user `9d2bfa61bb7245579090eb060baf69ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:09.493245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8590a9df-6c68-43a0-9c95-ddff22042497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.546422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be89c8-f48a-4da3-9892-cdfa5c21e428 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:09.549384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-75be89c8-f48a-4da3-9892-cdfa5c21e428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.568660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be89c8-f48a-4da3-9892-cdfa5c21e428 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:09.573857 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:09.574881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.575305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.575883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] PATH_INFO: `/v3/users/e21616bf7b5c44e6ad6a8c4a9cc6d8af` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.577384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e21616bf7b5c44e6ad6a8c4a9cc6d8af)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:09.579808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:09.592437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1062/2141] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:27:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:09.606662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590a9df-6c68-43a0-9c95-ddff22042497 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:09.609289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8590a9df-6c68-43a0-9c95-ddff22042497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.640508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e21616bf7b5c44e6ad6a8c4a9cc6d8af'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.641344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e21616bf7b5c44e6ad6a8c4a9cc6d8af'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.642010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590a9df-6c68-43a0-9c95-ddff22042497 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:09.651273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:09.652774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.653600 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.655237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.659965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.667908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e21616bf7b5c44e6ad6a8c4a9cc6d8af'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.679684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:09.681920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:09.692922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1063/2142] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:27:09.699976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e21616bf7b5c44e6ad6a8c4a9cc6d8af'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.701919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e21616bf7b5c44e6ad6a8c4a9cc6d8af'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:09.729347 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1080/2143] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:09 2022] DELETE /identity/v3/users/e21616bf7b5c44e6ad6a8c4a9cc6d8af => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:09.856851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:09.858760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.874409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb338b5-731d-453d-ae70-0543c79404aa service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:09.919723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:09.921281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:09.971036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:09.972697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:09.972697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:09.972697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] PATH_INFO: `/v3/projects/c2dfa48915864f7dbadea79f5b38e997` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:09.973779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2dfa48915864f7dbadea79f5b38e997)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:09.975740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.003438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b3562-fceb-4dd5-8837-dba5d6254d0f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.008268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.009639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.009639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.009843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] PATH_INFO: `/v3/projects/24fc772682c740e69c5b49ce8ec7d8e9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.011470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24fc772682c740e69c5b49ce8ec7d8e9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.016422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.038166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:10.039857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2dfa48915864f7dbadea79f5b38e997', 'request_id': 'req-b92cc8c4-dc71-48fb-977e-5502722f269e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.052588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2dfa48915864f7dbadea79f5b38e997', 'request_id': 'req-b92cc8c4-dc71-48fb-977e-5502722f269e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.063529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:10.066836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2dfa48915864f7dbadea79f5b38e997', 'request_id': 'req-b92cc8c4-dc71-48fb-977e-5502722f269e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.079001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24fc772682c740e69c5b49ce8ec7d8e9', 'request_id': 'req-a28054f2-e356-4076-a5ec-112ae35f10f5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.080556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] The token cache is being invalidate because project c2dfa48915864f7dbadea79f5b38e997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:10.080931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.082045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.083595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1064/2144] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:09 2022] DELETE /identity/v3/projects/c2dfa48915864f7dbadea79f5b38e997 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:10.084419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.085772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:10.090165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24fc772682c740e69c5b49ce8ec7d8e9', 'request_id': 'req-a28054f2-e356-4076-a5ec-112ae35f10f5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.097064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24fc772682c740e69c5b49ce8ec7d8e9', 'request_id': 'req-a28054f2-e356-4076-a5ec-112ae35f10f5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.114038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] The token cache is being invalidate because project 24fc772682c740e69c5b49ce8ec7d8e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:10.114327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.115607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a28054f2-e356-4076-a5ec-112ae35f10f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.117374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1081/2145] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:09 2022] DELETE /identity/v3/projects/24fc772682c740e69c5b49ce8ec7d8e9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:10.118568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e6acc-8532-4bb1-8721-5d9c9970357d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.118849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e6acc-8532-4bb1-8721-5d9c9970357d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.119123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e6acc-8532-4bb1-8721-5d9c9970357d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.142311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-891e6acc-8532-4bb1-8721-5d9c9970357d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:10.143558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-891e6acc-8532-4bb1-8721-5d9c9970357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:10.168628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc8c4-dc71-48fb-977e-5502722f269e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.207541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1082/2146] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:10.209144 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71003d-c042-4740-9f5b-182576ab5263 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.209503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71003d-c042-4740-9f5b-182576ab5263 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.209826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71003d-c042-4740-9f5b-182576ab5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.231734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ec71003d-c042-4740-9f5b-182576ab5263 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:10.233135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ec71003d-c042-4740-9f5b-182576ab5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:10.241695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1083/2147] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:10.242755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c17d23-c124-4002-b89a-5ad421ef0ed4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.243044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c17d23-c124-4002-b89a-5ad421ef0ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.243318 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c17d23-c124-4002-b89a-5ad421ef0ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.247035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.247937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.248282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.248754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] PATH_INFO: `/v3/projects/1a433429eb484bd4b1dae47092f38c2a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.249992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a433429eb484bd4b1dae47092f38c2a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.251962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.264680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-35c17d23-c124-4002-b89a-5ad421ef0ed4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:10.265881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-35c17d23-c124-4002-b89a-5ad421ef0ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:10.274069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1084/2148] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:10.275282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452c5b1-ffee-4906-ad46-e473a09c930e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.275650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452c5b1-ffee-4906-ad46-e473a09c930e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.275878 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452c5b1-ffee-4906-ad46-e473a09c930e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.295118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6452c5b1-ffee-4906-ad46-e473a09c930e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:10.296496 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6452c5b1-ffee-4906-ad46-e473a09c930e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:10.304066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a433429eb484bd4b1dae47092f38c2a', 'request_id': 'req-f30107bb-eeb7-4057-86b1-21920bca4162'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.311964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a433429eb484bd4b1dae47092f38c2a', 'request_id': 'req-f30107bb-eeb7-4057-86b1-21920bca4162'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.318226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a433429eb484bd4b1dae47092f38c2a', 'request_id': 'req-f30107bb-eeb7-4057-86b1-21920bca4162'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.331705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] The token cache is being invalidate because project 1a433429eb484bd4b1dae47092f38c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:10.331994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.332720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:10.334539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1065/2149] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:10 2022] DELETE /identity/v3/projects/1a433429eb484bd4b1dae47092f38c2a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:10.338683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.340103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:10.358674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:10.373530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1085/2150] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:10.374916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efbc82-ed60-4948-95ef-1d97c8912753 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.375274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efbc82-ed60-4948-95ef-1d97c8912753 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.375636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efbc82-ed60-4948-95ef-1d97c8912753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.393580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b8efbc82-ed60-4948-95ef-1d97c8912753 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:10.394761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b8efbc82-ed60-4948-95ef-1d97c8912753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:10.403082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1086/2151] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:10.403982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efbc82-ed60-4948-95ef-1d97c8912753 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.414027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efbc82-ed60-4948-95ef-1d97c8912753 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.418610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066df769-8c2b-4221-b1ba-c3686455f2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.419249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066df769-8c2b-4221-b1ba-c3686455f2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.419567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066df769-8c2b-4221-b1ba-c3686455f2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.419845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066df769-8c2b-4221-b1ba-c3686455f2ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.420372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30107bb-eeb7-4057-86b1-21920bca4162 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.420645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066df769-8c2b-4221-b1ba-c3686455f2ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:27:10.420954 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066df769-8c2b-4221-b1ba-c3686455f2ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.422546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066df769-8c2b-4221-b1ba-c3686455f2ca admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.427722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.428412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.428689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.428958 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.429703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:27:10.430762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.432894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.436973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1087/2152] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:27:10.441669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139e88-9455-463f-b1e1-e7d957a16d3f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.441998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139e88-9455-463f-b1e1-e7d957a16d3f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.442343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139e88-9455-463f-b1e1-e7d957a16d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.445342 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1066/2153] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:27:10.446026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.456113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc7d0-8787-4536-b4a0-484f5ed87a54 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.460922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d4eb6-0e2f-4fb2-87db-4823d79c2652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.461688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d4eb6-0e2f-4fb2-87db-4823d79c2652 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.462068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d4eb6-0e2f-4fb2-87db-4823d79c2652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.462480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d4eb6-0e2f-4fb2-87db-4823d79c2652 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.462601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f5139e88-9455-463f-b1e1-e7d957a16d3f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:10.463363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d4eb6-0e2f-4fb2-87db-4823d79c2652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.464010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f5139e88-9455-463f-b1e1-e7d957a16d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:10.465330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d4eb6-0e2f-4fb2-87db-4823d79c2652 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.473952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1088/2154] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:10.474735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5139e88-9455-463f-b1e1-e7d957a16d3f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.485838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5139e88-9455-463f-b1e1-e7d957a16d3f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.503431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f0641-f4a4-4e91-8ff4-5d23322a1934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.504110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f0641-f4a4-4e91-8ff4-5d23322a1934 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.504391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f0641-f4a4-4e91-8ff4-5d23322a1934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.507703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1067/2155] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:10.507882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f0641-f4a4-4e91-8ff4-5d23322a1934 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.507989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d4eb6-0e2f-4fb2-87db-4823d79c2652 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.508088 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f0641-f4a4-4e91-8ff4-5d23322a1934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.508088 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f0641-f4a4-4e91-8ff4-5d23322a1934 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.544416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1089/2156] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:10.547463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f0641-f4a4-4e91-8ff4-5d23322a1934 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.553921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:10.580097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d4eb6-0e2f-4fb2-87db-4823d79c2652 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.616143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f0641-f4a4-4e91-8ff4-5d23322a1934 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.620830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7359d-249b-4c2c-8071-724863118833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.621747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7359d-249b-4c2c-8071-724863118833 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.622021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7359d-249b-4c2c-8071-724863118833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.622292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7359d-249b-4c2c-8071-724863118833 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.623003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7359d-249b-4c2c-8071-724863118833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.624992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7359d-249b-4c2c-8071-724863118833 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.638258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.639518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.639518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.639709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.640458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:27:10.640771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.642407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.648826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1090/2157] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:10.648826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7359d-249b-4c2c-8071-724863118833 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.660055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1068/2158] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:27:10.660333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7359d-249b-4c2c-8071-724863118833 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.661388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.665329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45942ddd-4848-40fe-b2fa-c90b2b53d13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.666001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45942ddd-4848-40fe-b2fa-c90b2b53d13f admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.666283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45942ddd-4848-40fe-b2fa-c90b2b53d13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.666571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45942ddd-4848-40fe-b2fa-c90b2b53d13f admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.667257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45942ddd-4848-40fe-b2fa-c90b2b53d13f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.669228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45942ddd-4848-40fe-b2fa-c90b2b53d13f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.672764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbdb62-935f-46da-af04-c2bf1889a526 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.677352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43fd30-6c95-41f7-9d01-e66fd03f9677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.678051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43fd30-6c95-41f7-9d01-e66fd03f9677 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.678335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43fd30-6c95-41f7-9d01-e66fd03f9677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.678614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43fd30-6c95-41f7-9d01-e66fd03f9677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.679361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43fd30-6c95-41f7-9d01-e66fd03f9677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.680927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43fd30-6c95-41f7-9d01-e66fd03f9677 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.693966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1091/2159] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:10.693966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45942ddd-4848-40fe-b2fa-c90b2b53d13f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.694256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1069/2160] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:10.696506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43fd30-6c95-41f7-9d01-e66fd03f9677 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.701095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45942ddd-4848-40fe-b2fa-c90b2b53d13f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.705961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e56c24-51bc-44c2-a214-31e35b1b668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.706642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e56c24-51bc-44c2-a214-31e35b1b668b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.706923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e56c24-51bc-44c2-a214-31e35b1b668b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.707092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43fd30-6c95-41f7-9d01-e66fd03f9677 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.707186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e56c24-51bc-44c2-a214-31e35b1b668b admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.707909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e56c24-51bc-44c2-a214-31e35b1b668b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.709852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e56c24-51bc-44c2-a214-31e35b1b668b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.716692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044b6d-4348-4a62-ae9a-30de9ba7ce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.718220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044b6d-4348-4a62-ae9a-30de9ba7ce45 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.718667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044b6d-4348-4a62-ae9a-30de9ba7ce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.718948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044b6d-4348-4a62-ae9a-30de9ba7ce45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.719718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3044b6d-4348-4a62-ae9a-30de9ba7ce45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.721644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3044b6d-4348-4a62-ae9a-30de9ba7ce45 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.732187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1070/2161] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:27:10.733710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044b6d-4348-4a62-ae9a-30de9ba7ce45 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.740428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:10.743078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1092/2162] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:10.743886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e56c24-51bc-44c2-a214-31e35b1b668b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.745016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044b6d-4348-4a62-ae9a-30de9ba7ce45 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.814880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be143f-2c09-438f-8f8a-dd73577203f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.815504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be143f-2c09-438f-8f8a-dd73577203f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.815787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be143f-2c09-438f-8f8a-dd73577203f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.816059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be143f-2c09-438f-8f8a-dd73577203f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.816995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be143f-2c09-438f-8f8a-dd73577203f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.818927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be143f-2c09-438f-8f8a-dd73577203f2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.820644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e56c24-51bc-44c2-a214-31e35b1b668b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.825575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfe71f-b2bb-4995-84fc-e1c63366271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.826842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfe71f-b2bb-4995-84fc-e1c63366271c admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.827138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfe71f-b2bb-4995-84fc-e1c63366271c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.827444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfe71f-b2bb-4995-84fc-e1c63366271c admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.828211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfe71f-b2bb-4995-84fc-e1c63366271c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.829865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfe71f-b2bb-4995-84fc-e1c63366271c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.838465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1071/2163] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:10.839499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be143f-2c09-438f-8f8a-dd73577203f2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.842645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1093/2164] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:10.843474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfe71f-b2bb-4995-84fc-e1c63366271c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.853531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfe71f-b2bb-4995-84fc-e1c63366271c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.853814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be143f-2c09-438f-8f8a-dd73577203f2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.858336 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef0798-89e7-406f-a044-387da9bef195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.858588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d386ec7-6b92-469d-9e52-f9d07a9b9ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.859258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d386ec7-6b92-469d-9e52-f9d07a9b9ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.859575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d386ec7-6b92-469d-9e52-f9d07a9b9ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.859666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef0798-89e7-406f-a044-387da9bef195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.859951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef0798-89e7-406f-a044-387da9bef195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.860053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d386ec7-6b92-469d-9e52-f9d07a9b9ac0 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.860280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef0798-89e7-406f-a044-387da9bef195 admin admin] PATH_INFO: `/v3/projects/589e1906e2d0471d8f1cf677d3f5d41a/users/c855c78b08b145fba6590767356578ad/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.860587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d386ec7-6b92-469d-9e52-f9d07a9b9ac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.862708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d386ec7-6b92-469d-9e52-f9d07a9b9ac0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.882125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef0798-89e7-406f-a044-387da9bef195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=589e1906e2d0471d8f1cf677d3f5d41a, user_id=c855c78b08b145fba6590767356578ad, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.885312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef0798-89e7-406f-a044-387da9bef195 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.888623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1072/2165] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:10 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:10.888623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d386ec7-6b92-469d-9e52-f9d07a9b9ac0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.897525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d386ec7-6b92-469d-9e52-f9d07a9b9ac0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.899842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1094/2166] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:10 2022] PUT /identity/v3/projects/589e1906e2d0471d8f1cf677d3f5d41a/users/c855c78b08b145fba6590767356578ad/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:10.900719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef0798-89e7-406f-a044-387da9bef195 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:10.936837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:10.965574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1c38c-dcd6-47e7-ad2f-00656601688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.966476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c38c-dcd6-47e7-ad2f-00656601688e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.966783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c38c-dcd6-47e7-ad2f-00656601688e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.967416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c38c-dcd6-47e7-ad2f-00656601688e admin admin] PATH_INFO: `/v3/projects/aacfe2d2bf374fcfb793d0278a9c8a81/users/bf836b07e2564a2e8650e6319ad0feed/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.972481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef0798-89e7-406f-a044-387da9bef195 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:10.981907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd18f2-437b-4c9e-8160-23293b2288e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:10.983999 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1c38c-dcd6-47e7-ad2f-00656601688e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aacfe2d2bf374fcfb793d0278a9c8a81, user_id=bf836b07e2564a2e8650e6319ad0feed, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.984779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd18f2-437b-4c9e-8160-23293b2288e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:10.985066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd18f2-437b-4c9e-8160-23293b2288e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:10.985343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd18f2-437b-4c9e-8160-23293b2288e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:10.986108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd18f2-437b-4c9e-8160-23293b2288e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:10.986693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1c38c-dcd6-47e7-ad2f-00656601688e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:10.987739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd18f2-437b-4c9e-8160-23293b2288e3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.000029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1095/2167] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:11.001290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a245f2-be25-4918-9c7d-cc01eb8219f1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.001579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a245f2-be25-4918-9c7d-cc01eb8219f1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.001847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a245f2-be25-4918-9c7d-cc01eb8219f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.004181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1073/2168] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:10 2022] PUT /identity/v3/projects/aacfe2d2bf374fcfb793d0278a9c8a81/users/bf836b07e2564a2e8650e6319ad0feed/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:11.005005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1c38c-dcd6-47e7-ad2f-00656601688e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.030924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b1a245f2-be25-4918-9c7d-cc01eb8219f1 None None] MFA Rules not processed for user `c855c78b08b145fba6590767356578ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:11.032159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b1a245f2-be25-4918-9c7d-cc01eb8219f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:11.069384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1c38c-dcd6-47e7-ad2f-00656601688e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.074104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7660ae-2569-4761-8ab9-90b78d3b297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.074790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7660ae-2569-4761-8ab9-90b78d3b297a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.075061 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7660ae-2569-4761-8ab9-90b78d3b297a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.075364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7660ae-2569-4761-8ab9-90b78d3b297a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.076110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7660ae-2569-4761-8ab9-90b78d3b297a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.077736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7660ae-2569-4761-8ab9-90b78d3b297a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.087851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1074/2169] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:11.089331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cbd89-09fc-4c79-8b5e-ed3f0c6fbb65 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.089630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cbd89-09fc-4c79-8b5e-ed3f0c6fbb65 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.089914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cbd89-09fc-4c79-8b5e-ed3f0c6fbb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.096802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:11.103096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1096/2170] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:11.108250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a245f2-be25-4918-9c7d-cc01eb8219f1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.118410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a245f2-be25-4918-9c7d-cc01eb8219f1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.121860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1f4cbd89-09fc-4c79-8b5e-ed3f0c6fbb65 None None] MFA Rules not processed for user `bf836b07e2564a2e8650e6319ad0feed`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:11.123736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f4cbd89-09fc-4c79-8b5e-ed3f0c6fbb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:11.124452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.125341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.125629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.125973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] PATH_INFO: `/v3/projects/afaaf8bde7134efc80c77d707bb309bc/users/710b406ba65646ed94c3f9d356b161f1/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.141448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afaaf8bde7134efc80c77d707bb309bc, user_id=710b406ba65646ed94c3f9d356b161f1, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.144156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.157344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1097/2171] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:11 2022] PUT /identity/v3/projects/afaaf8bde7134efc80c77d707bb309bc/users/710b406ba65646ed94c3f9d356b161f1/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:11.158146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.159782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:11.192801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1075/2172] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:11.194090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34c921-f50d-4b4e-9f01-cdc5b7ffb071 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.194382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34c921-f50d-4b4e-9f01-cdc5b7ffb071 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.194662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34c921-f50d-4b4e-9f01-cdc5b7ffb071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.212140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8b34c921-f50d-4b4e-9f01-cdc5b7ffb071 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:11.213494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8b34c921-f50d-4b4e-9f01-cdc5b7ffb071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:11.222664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907156e-a0e6-4494-ab83-0e8b32343980 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.226885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.227783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.228067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.228338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.280348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1076/2173] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:11.281258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34c921-f50d-4b4e-9f01-cdc5b7ffb071 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.285726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:11.294156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34c921-f50d-4b4e-9f01-cdc5b7ffb071 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.300914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03994e4a-3d4b-4d9f-9528-0224e65ef089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.301590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03994e4a-3d4b-4d9f-9528-0224e65ef089 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.301879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03994e4a-3d4b-4d9f-9528-0224e65ef089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.302150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03994e4a-3d4b-4d9f-9528-0224e65ef089 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.302878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03994e4a-3d4b-4d9f-9528-0224e65ef089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.304983 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03994e4a-3d4b-4d9f-9528-0224e65ef089 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.305525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.306961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.318304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1098/2174] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:11.319071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.321360 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:11.340131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1077/2175] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:11 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:11.344513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03994e4a-3d4b-4d9f-9528-0224e65ef089 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.419562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce57767-46eb-4064-9c40-b90c1d2274de service glance] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.429436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03994e4a-3d4b-4d9f-9528-0224e65ef089 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.438679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebfbe0-f784-41c4-a28e-1f67c506af23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.439402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebfbe0-f784-41c4-a28e-1f67c506af23 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.439688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebfbe0-f784-41c4-a28e-1f67c506af23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.439967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebfbe0-f784-41c4-a28e-1f67c506af23 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.440673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ebfbe0-f784-41c4-a28e-1f67c506af23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.443191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ebfbe0-f784-41c4-a28e-1f67c506af23 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.462664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:11.470973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1078/2176] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:11 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:11.475273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebfbe0-f784-41c4-a28e-1f67c506af23 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.485992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebfbe0-f784-41c4-a28e-1f67c506af23 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.487433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e0b0d-c1b2-402d-b5c9-b250354e6648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.488092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e0b0d-c1b2-402d-b5c9-b250354e6648 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.488376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e0b0d-c1b2-402d-b5c9-b250354e6648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.488745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e0b0d-c1b2-402d-b5c9-b250354e6648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.491238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd8904-f1c8-4643-b38d-b8e849e2425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.492022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd8904-f1c8-4643-b38d-b8e849e2425c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.492406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd8904-f1c8-4643-b38d-b8e849e2425c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.492773 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd8904-f1c8-4643-b38d-b8e849e2425c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.493774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd8904-f1c8-4643-b38d-b8e849e2425c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.495378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd8904-f1c8-4643-b38d-b8e849e2425c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.504847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e0b0d-c1b2-402d-b5c9-b250354e6648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.507262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e0b0d-c1b2-402d-b5c9-b250354e6648 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.509018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1079/2177] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:27:11.512668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd8904-f1c8-4643-b38d-b8e849e2425c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.520321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1099/2178] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:11.527115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd8904-f1c8-4643-b38d-b8e849e2425c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.533653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f4018-923d-4ba8-8dd1-6a9e7a87a1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.534505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f4018-923d-4ba8-8dd1-6a9e7a87a1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.534838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f4018-923d-4ba8-8dd1-6a9e7a87a1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.535142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f4018-923d-4ba8-8dd1-6a9e7a87a1a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.536454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f4018-923d-4ba8-8dd1-6a9e7a87a1a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.538271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f4018-923d-4ba8-8dd1-6a9e7a87a1a1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.555110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1080/2179] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:11.559718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e0b0d-c1b2-402d-b5c9-b250354e6648 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.570118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e0b0d-c1b2-402d-b5c9-b250354e6648 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.576426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401068-6af0-4fdd-9bff-33bf29ec5b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.577344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401068-6af0-4fdd-9bff-33bf29ec5b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.577758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401068-6af0-4fdd-9bff-33bf29ec5b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.578068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401068-6af0-4fdd-9bff-33bf29ec5b52 admin admin] PATH_INFO: `/v3/projects/c61b54aae8794b6e8af417feacdc6fbb/users/249f83f112b14f5f9168d45a4389cd18/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.601155 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff401068-6af0-4fdd-9bff-33bf29ec5b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61b54aae8794b6e8af417feacdc6fbb, user_id=249f83f112b14f5f9168d45a4389cd18, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.604216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff401068-6af0-4fdd-9bff-33bf29ec5b52 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.620063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1100/2180] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:11 2022] PUT /identity/v3/projects/c61b54aae8794b6e8af417feacdc6fbb/users/249f83f112b14f5f9168d45a4389cd18/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:11.633007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b829cf-fb1a-4e2f-9a2c-6e2a65c6b086 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.633342 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b829cf-fb1a-4e2f-9a2c-6e2a65c6b086 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.633665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b829cf-fb1a-4e2f-9a2c-6e2a65c6b086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.668615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f2b829cf-fb1a-4e2f-9a2c-6e2a65c6b086 None None] MFA Rules not processed for user `249f83f112b14f5f9168d45a4389cd18`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:11.670568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f2b829cf-fb1a-4e2f-9a2c-6e2a65c6b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:11.766597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1081/2181] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:11.771481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401068-6af0-4fdd-9bff-33bf29ec5b52 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:11.915608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401068-6af0-4fdd-9bff-33bf29ec5b52 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:11.919399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab70b24-601a-47fe-b4de-c7e60a5153ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:11.920878 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab70b24-601a-47fe-b4de-c7e60a5153ee admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:11.921495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab70b24-601a-47fe-b4de-c7e60a5153ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:11.921495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab70b24-601a-47fe-b4de-c7e60a5153ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:11.922452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab70b24-601a-47fe-b4de-c7e60a5153ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:11.924559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab70b24-601a-47fe-b4de-c7e60a5153ee admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:11.980402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1101/2182] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:11 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:11.981960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b829cf-fb1a-4e2f-9a2c-6e2a65c6b086 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:12.003277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b829cf-fb1a-4e2f-9a2c-6e2a65c6b086 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:12.070808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532fc55-cf33-452c-ab70-7a971c8346a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:12.071828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532fc55-cf33-452c-ab70-7a971c8346a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:12.072109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532fc55-cf33-452c-ab70-7a971c8346a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:12.072387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532fc55-cf33-452c-ab70-7a971c8346a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:12.073084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532fc55-cf33-452c-ab70-7a971c8346a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:12.075215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532fc55-cf33-452c-ab70-7a971c8346a1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:12.099032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1082/2183] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:11 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:12.100152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab70b24-601a-47fe-b4de-c7e60a5153ee admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:12.106842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:12.111619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab70b24-601a-47fe-b4de-c7e60a5153ee admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:12.118089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4e5e-27da-4406-af20-98e509c501b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:12.118758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4e5e-27da-4406-af20-98e509c501b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:12.119036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4e5e-27da-4406-af20-98e509c501b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:12.119310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4e5e-27da-4406-af20-98e509c501b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:12.120069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb4e5e-27da-4406-af20-98e509c501b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:12.121922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb4e5e-27da-4406-af20-98e509c501b4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:12.134044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1102/2184] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:27:12.138408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532fc55-cf33-452c-ab70-7a971c8346a1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:12.148246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532fc55-cf33-452c-ab70-7a971c8346a1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:12.152792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1518b5-127a-45b7-913c-7381c8a4e5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:12.153664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1518b5-127a-45b7-913c-7381c8a4e5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:12.153942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1518b5-127a-45b7-913c-7381c8a4e5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:12.154214 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1518b5-127a-45b7-913c-7381c8a4e5f7 admin admin] PATH_INFO: `/v3/projects/e027048191e141deb3b78f6a49fe031f/users/326f9726bfd84ba180b7a80ee20b32ad/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:12.170805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1518b5-127a-45b7-913c-7381c8a4e5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e027048191e141deb3b78f6a49fe031f, user_id=326f9726bfd84ba180b7a80ee20b32ad, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:12.173682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1518b5-127a-45b7-913c-7381c8a4e5f7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:12.187982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1083/2185] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:12 2022] PUT /identity/v3/projects/e027048191e141deb3b78f6a49fe031f/users/326f9726bfd84ba180b7a80ee20b32ad/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:12.191921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4e5e-27da-4406-af20-98e509c501b4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:12.204960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4e5e-27da-4406-af20-98e509c501b4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:12.274040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855f489-605e-4eb7-bc5a-4842246297ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:12.274730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855f489-605e-4eb7-bc5a-4842246297ae admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:12.275325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855f489-605e-4eb7-bc5a-4842246297ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:12.275325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855f489-605e-4eb7-bc5a-4842246297ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:12.276220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855f489-605e-4eb7-bc5a-4842246297ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:12.277899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855f489-605e-4eb7-bc5a-4842246297ae admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:12.290631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1103/2186] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:12.294917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1518b5-127a-45b7-913c-7381c8a4e5f7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:12.309857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1518b5-127a-45b7-913c-7381c8a4e5f7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:12.316172 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:12.317159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:12.317352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:12.317628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] PATH_INFO: `/v3/projects/e027048191e141deb3b78f6a49fe031f/users/326f9726bfd84ba180b7a80ee20b32ad/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:12.319657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e027048191e141deb3b78f6a49fe031f, user_id=326f9726bfd84ba180b7a80ee20b32ad, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:12.322431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:12.334497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1084/2187] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:12 2022] PUT /identity/v3/projects/e027048191e141deb3b78f6a49fe031f/users/326f9726bfd84ba180b7a80ee20b32ad/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:12.347919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148866-734c-4976-b04b-85a5f176f540 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:12.348212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148866-734c-4976-b04b-85a5f176f540 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:12.348491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148866-734c-4976-b04b-85a5f176f540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:12.381409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-db148866-734c-4976-b04b-85a5f176f540 None None] MFA Rules not processed for user `326f9726bfd84ba180b7a80ee20b32ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:12.382709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-db148866-734c-4976-b04b-85a5f176f540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:12.499695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1104/2188] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:12.508052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:12.510317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:12.541889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:12.605147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee97c1-a10e-4b38-8ce5-a99c40fc2570 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:12.610227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd32e00-30d7-48e8-bc4b-669524e123e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:12.611488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32e00-30d7-48e8-bc4b-669524e123e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:12.611488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32e00-30d7-48e8-bc4b-669524e123e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:12.611850 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32e00-30d7-48e8-bc4b-669524e123e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:12.677908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd32e00-30d7-48e8-bc4b-669524e123e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:12.679567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd32e00-30d7-48e8-bc4b-669524e123e0 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:12.691000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1085/2189] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:27:15.780861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db148866-734c-4976-b04b-85a5f176f540 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:15.802987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db148866-734c-4976-b04b-85a5f176f540 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:15.807641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4554-2949-4457-b751-6ab7f4e25792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:15.808634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4554-2949-4457-b751-6ab7f4e25792 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:15.808992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4554-2949-4457-b751-6ab7f4e25792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:15.809366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4554-2949-4457-b751-6ab7f4e25792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:15.906349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4554-2949-4457-b751-6ab7f4e25792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:15.907932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4554-2949-4457-b751-6ab7f4e25792 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:15.918406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1105/2190] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:27:16.872668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0550f6-1ddd-43d4-a56c-b03c48151a30 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:16.874528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0550f6-1ddd-43d4-a56c-b03c48151a30 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:16.874986 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0550f6-1ddd-43d4-a56c-b03c48151a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:16.928019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3d0550f6-1ddd-43d4-a56c-b03c48151a30 None None] MFA Rules not processed for user `710b406ba65646ed94c3f9d356b161f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:16.930121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3d0550f6-1ddd-43d4-a56c-b03c48151a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:16.983635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4554-2949-4457-b751-6ab7f4e25792 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:17.018238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4554-2949-4457-b751-6ab7f4e25792 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:17.023456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:17.024433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:17.025256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:17.025807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:17.039540 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:17.041757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:17.048266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:17.061879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1106/2191] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:17.090860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1086/2192] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:17.107564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:17.109650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:17.280514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcf8b5-b7ce-419e-996b-e71e04b0909f service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:17.287739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357ef94-7147-4e4b-a970-07ac8fb4d300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:17.288415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357ef94-7147-4e4b-a970-07ac8fb4d300 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:17.289235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357ef94-7147-4e4b-a970-07ac8fb4d300 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:17.289235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357ef94-7147-4e4b-a970-07ac8fb4d300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:17.299077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357ef94-7147-4e4b-a970-07ac8fb4d300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:17.301039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357ef94-7147-4e4b-a970-07ac8fb4d300 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:17.313847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1107/2193] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:17.706673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0550f6-1ddd-43d4-a56c-b03c48151a30 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:17.708726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3d0550f6-1ddd-43d4-a56c-b03c48151a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:17.793665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0550f6-1ddd-43d4-a56c-b03c48151a30 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:17.799694 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:17.800619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:17.800905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:17.801184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] PATH_INFO: `/v3/users/5e167442a2274b9c8f4ddc799c15c0ea` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:17.802709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e167442a2274b9c8f4ddc799c15c0ea)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:17.805138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:17.810102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:17.867961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e167442a2274b9c8f4ddc799c15c0ea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:17.879876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e167442a2274b9c8f4ddc799c15c0ea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:17.909377 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e167442a2274b9c8f4ddc799c15c0ea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:17.910141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e167442a2274b9c8f4ddc799c15c0ea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:17.920520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e167442a2274b9c8f4ddc799c15c0ea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:17.926912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1087/2194] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:17 2022] DELETE /identity/v3/users/5e167442a2274b9c8f4ddc799c15c0ea => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:17.931850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3adee-bb1d-43d1-ac2e-9f4e49f3e522 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:17.934265 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3adee-bb1d-43d1-ac2e-9f4e49f3e522 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:17.934636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3adee-bb1d-43d1-ac2e-9f4e49f3e522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:17.958866 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f6f3adee-bb1d-43d1-ac2e-9f4e49f3e522 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:17.960308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f6f3adee-bb1d-43d1-ac2e-9f4e49f3e522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:18.035516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1108/2195] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:18.042955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:18.046148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:18.130282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc86ed-7d06-4c60-acff-bddedf87afbf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:18.134050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193ff20-7a73-42a9-b56c-ffe7118c913e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:18.134708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ff20-7a73-42a9-b56c-ffe7118c913e service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:18.134990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ff20-7a73-42a9-b56c-ffe7118c913e service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:18.135299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ff20-7a73-42a9-b56c-ffe7118c913e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:18.145376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193ff20-7a73-42a9-b56c-ffe7118c913e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:18.146795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193ff20-7a73-42a9-b56c-ffe7118c913e service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:18.158955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1088/2196] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:18.541073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3adee-bb1d-43d1-ac2e-9f4e49f3e522 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:18.542598 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f6f3adee-bb1d-43d1-ac2e-9f4e49f3e522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:18.571797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3adee-bb1d-43d1-ac2e-9f4e49f3e522 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:18.588563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:18.589769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:18.590260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:18.590724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] PATH_INFO: `/v3/projects/c62b14b879914ef18d78381ff77ed5c2` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:18.592312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62b14b879914ef18d78381ff77ed5c2)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:18.594690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:18.687691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62b14b879914ef18d78381ff77ed5c2', 'request_id': 'req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:18.705300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62b14b879914ef18d78381ff77ed5c2', 'request_id': 'req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:18.741748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62b14b879914ef18d78381ff77ed5c2', 'request_id': 'req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:18.787448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] The token cache is being invalidate because project c62b14b879914ef18d78381ff77ed5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:18.787750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:18.788435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e977b980-7d0e-4b1a-bb84-e55c96dad6d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:18.790027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1109/2197] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:18 2022] DELETE /identity/v3/projects/c62b14b879914ef18d78381ff77ed5c2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:18.794487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193ff20-7a73-42a9-b56c-ffe7118c913e service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:18.814807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193ff20-7a73-42a9-b56c-ffe7118c913e service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:18.910792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:18.911898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:18.912854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:18.913410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] PATH_INFO: `/v3/users/e5dc182c11ae4b83872a165f60f49084` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:18.916453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5dc182c11ae4b83872a165f60f49084)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:18.920665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:18.972132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:19.002994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5dc182c11ae4b83872a165f60f49084'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.013509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5dc182c11ae4b83872a165f60f49084'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.036082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5dc182c11ae4b83872a165f60f49084'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.038332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5dc182c11ae4b83872a165f60f49084'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.048935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5dc182c11ae4b83872a165f60f49084'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.056135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1089/2198] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:18 2022] DELETE /identity/v3/users/e5dc182c11ae4b83872a165f60f49084 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:19.239397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267815b-9146-40af-8c54-b57f5f4e52d9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:19.240361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267815b-9146-40af-8c54-b57f5f4e52d9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:19.240875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267815b-9146-40af-8c54-b57f5f4e52d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:19.260323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:19.268749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6267815b-9146-40af-8c54-b57f5f4e52d9 None None] MFA Rules not processed for user `710b406ba65646ed94c3f9d356b161f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:19.270227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6267815b-9146-40af-8c54-b57f5f4e52d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:19.379697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1110/2199] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:19.430072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:19.436519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:19.520636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e910d0-3fd8-4a1b-b74e-13d6c34d7997 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:19.525346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:19.526243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:19.526521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:19.526807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] PATH_INFO: `/v3/projects/b411a6745aef4de1bb6f91d7be1206f7` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:19.527993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b411a6745aef4de1bb6f91d7be1206f7)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:19.530668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:19.591646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b411a6745aef4de1bb6f91d7be1206f7', 'request_id': 'req-f7c56ac1-f28a-4e32-a173-6ee271c5832f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.602239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b411a6745aef4de1bb6f91d7be1206f7', 'request_id': 'req-f7c56ac1-f28a-4e32-a173-6ee271c5832f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.609907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b411a6745aef4de1bb6f91d7be1206f7', 'request_id': 'req-f7c56ac1-f28a-4e32-a173-6ee271c5832f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.628791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] The token cache is being invalidate because project b411a6745aef4de1bb6f91d7be1206f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:19.629097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.629772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f7c56ac1-f28a-4e32-a173-6ee271c5832f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:19.631393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1090/2200] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:19 2022] DELETE /identity/v3/projects/b411a6745aef4de1bb6f91d7be1206f7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:19.636363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0202d-537b-40ba-abe9-da3526eb1a86 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:19.636657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0202d-537b-40ba-abe9-da3526eb1a86 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:19.636935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0202d-537b-40ba-abe9-da3526eb1a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:19.657286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-48b0202d-537b-40ba-abe9-da3526eb1a86 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:19.658829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-48b0202d-537b-40ba-abe9-da3526eb1a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:19.732026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1111/2201] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:19.740625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed510fd-e9c0-4d1f-b2b7-66a9005ee52d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:19.740990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed510fd-e9c0-4d1f-b2b7-66a9005ee52d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:19.741309 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed510fd-e9c0-4d1f-b2b7-66a9005ee52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:19.764280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5ed510fd-e9c0-4d1f-b2b7-66a9005ee52d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:19.765620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5ed510fd-e9c0-4d1f-b2b7-66a9005ee52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:19.774988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1091/2202] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:19.779876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0202d-537b-40ba-abe9-da3526eb1a86 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:19.790287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0202d-537b-40ba-abe9-da3526eb1a86 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:19.795390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:19.796352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:19.796702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:19.797107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:19.797921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:27:19.798308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:19.800034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:19.820563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1112/2203] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:27:19.824926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed510fd-e9c0-4d1f-b2b7-66a9005ee52d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:19.849348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed510fd-e9c0-4d1f-b2b7-66a9005ee52d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:19.857589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbe152-075a-4c3f-bfe8-a3f8e63c36c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:19.858990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbe152-075a-4c3f-bfe8-a3f8e63c36c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:19.861927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbe152-075a-4c3f-bfe8-a3f8e63c36c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:19.862378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbe152-075a-4c3f-bfe8-a3f8e63c36c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:19.863553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbe152-075a-4c3f-bfe8-a3f8e63c36c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:19.867243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbe152-075a-4c3f-bfe8-a3f8e63c36c4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:19.912028 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1092/2204] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:19 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:19.916822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:19.939873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d2238-85ba-46a7-9fc1-9dbf0ef56acc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:20.025752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b6fd5-81c6-4dc4-aa5e-766bc3e2675b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:20.026555 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6fd5-81c6-4dc4-aa5e-766bc3e2675b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:20.026961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6fd5-81c6-4dc4-aa5e-766bc3e2675b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:20.031864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6fd5-81c6-4dc4-aa5e-766bc3e2675b admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:20.032978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b6fd5-81c6-4dc4-aa5e-766bc3e2675b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:20.035182 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b6fd5-81c6-4dc4-aa5e-766bc3e2675b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:20.062410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1113/2205] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:19 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:20.066591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbe152-075a-4c3f-bfe8-a3f8e63c36c4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:20.080134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbe152-075a-4c3f-bfe8-a3f8e63c36c4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:20.086937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52489-cb22-46ab-8c41-61175ae9bec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:20.087798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52489-cb22-46ab-8c41-61175ae9bec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:20.088201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52489-cb22-46ab-8c41-61175ae9bec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:20.088585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52489-cb22-46ab-8c41-61175ae9bec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:20.089737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c52489-cb22-46ab-8c41-61175ae9bec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:20.092720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c52489-cb22-46ab-8c41-61175ae9bec9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:20.094842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:20.117921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1093/2206] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:20.121767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b6fd5-81c6-4dc4-aa5e-766bc3e2675b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:20.133763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b6fd5-81c6-4dc4-aa5e-766bc3e2675b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:20.139198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9989d2-4b9b-4da6-8208-271f7b59d86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:20.140080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9989d2-4b9b-4da6-8208-271f7b59d86f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:20.140384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9989d2-4b9b-4da6-8208-271f7b59d86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:20.140639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9989d2-4b9b-4da6-8208-271f7b59d86f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:20.141383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9989d2-4b9b-4da6-8208-271f7b59d86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:20.143021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9989d2-4b9b-4da6-8208-271f7b59d86f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:20.158586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1114/2207] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:27:20.160193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52489-cb22-46ab-8c41-61175ae9bec9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:20.171402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52489-cb22-46ab-8c41-61175ae9bec9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:20.180046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:20.181150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:20.182108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:20.182108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] PATH_INFO: `/v3/projects/1af8a82625604c13afd7ba465f05c9a0/users/73b0ac9c82e84e16904e3d8fde652f09/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:20.198402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af8a82625604c13afd7ba465f05c9a0, user_id=73b0ac9c82e84e16904e3d8fde652f09, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:20.201761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:20.220174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1094/2208] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:20 2022] PUT /identity/v3/projects/1af8a82625604c13afd7ba465f05c9a0/users/73b0ac9c82e84e16904e3d8fde652f09/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:20.230673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37743641-e67d-473c-93db-f90ea7324051 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:20.231007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37743641-e67d-473c-93db-f90ea7324051 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:20.231328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37743641-e67d-473c-93db-f90ea7324051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:20.270905 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-37743641-e67d-473c-93db-f90ea7324051 None None] MFA Rules not processed for user `73b0ac9c82e84e16904e3d8fde652f09`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:20.273374 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-37743641-e67d-473c-93db-f90ea7324051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:20.294403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:20.387607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1115/2209] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:20.402928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:20.406266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:20.537358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fead745-220c-42aa-992d-7e2a60b38bb6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:20.542912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cf227-9411-440a-b04a-e4e6a8dbf139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:20.543900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cf227-9411-440a-b04a-e4e6a8dbf139 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:20.544374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cf227-9411-440a-b04a-e4e6a8dbf139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:20.544662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cf227-9411-440a-b04a-e4e6a8dbf139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:20.555235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cf227-9411-440a-b04a-e4e6a8dbf139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:20.556768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cf227-9411-440a-b04a-e4e6a8dbf139 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:20.568745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1095/2210] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:27:20.594471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37743641-e67d-473c-93db-f90ea7324051 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:20.687183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37743641-e67d-473c-93db-f90ea7324051 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:20.691771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:20.692730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:20.693013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:20.693291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] PATH_INFO: `/v3/users/73b0ac9c82e84e16904e3d8fde652f09` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:20.694552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73b0ac9c82e84e16904e3d8fde652f09)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:20.697123 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:20.770781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73b0ac9c82e84e16904e3d8fde652f09'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:20.771723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73b0ac9c82e84e16904e3d8fde652f09'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:20.781042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73b0ac9c82e84e16904e3d8fde652f09'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:20.798010 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73b0ac9c82e84e16904e3d8fde652f09'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:20.798949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73b0ac9c82e84e16904e3d8fde652f09'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:20.819048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1116/2211] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:20 2022] DELETE /identity/v3/users/73b0ac9c82e84e16904e3d8fde652f09 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:20.823637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181ca9-e7f5-4c6f-8164-c1e0372620d4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:20.823928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181ca9-e7f5-4c6f-8164-c1e0372620d4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:20.824257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181ca9-e7f5-4c6f-8164-c1e0372620d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:20.845247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4f181ca9-e7f5-4c6f-8164-c1e0372620d4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:20.846820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4f181ca9-e7f5-4c6f-8164-c1e0372620d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:20.936802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1096/2212] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:20.947928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:20.951725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:21.006181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:21.036693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cf8e8-1724-4bab-ae6d-f1f2c5088442 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:21.040906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1e842-5729-451a-8ec7-2ca8c10fd54d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:21.041609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e842-5729-451a-8ec7-2ca8c10fd54d service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:21.041895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e842-5729-451a-8ec7-2ca8c10fd54d service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:21.042193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e842-5729-451a-8ec7-2ca8c10fd54d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:21.056281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1e842-5729-451a-8ec7-2ca8c10fd54d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:21.057846 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1e842-5729-451a-8ec7-2ca8c10fd54d service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:21.068705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1117/2213] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:21.414825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f181ca9-e7f5-4c6f-8164-c1e0372620d4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:21.416346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4f181ca9-e7f5-4c6f-8164-c1e0372620d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:21.430312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f181ca9-e7f5-4c6f-8164-c1e0372620d4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:21.442104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:21.443002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:21.443906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:21.444247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] PATH_INFO: `/v3/projects/1af8a82625604c13afd7ba465f05c9a0` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:21.445836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af8a82625604c13afd7ba465f05c9a0)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:21.448036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:21.520687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af8a82625604c13afd7ba465f05c9a0', 'request_id': 'req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.549461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af8a82625604c13afd7ba465f05c9a0', 'request_id': 'req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.567388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af8a82625604c13afd7ba465f05c9a0', 'request_id': 'req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.575431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:21.586432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] The token cache is being invalidate because project 1af8a82625604c13afd7ba465f05c9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:21.586777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.588087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.590403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1097/2214] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:21 2022] DELETE /identity/v3/projects/1af8a82625604c13afd7ba465f05c9a0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:21.597068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032b97-ec10-4f40-8faa-95ecee97b5ae None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:21.597901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032b97-ec10-4f40-8faa-95ecee97b5ae None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:21.598388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032b97-ec10-4f40-8faa-95ecee97b5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:21.616551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:21.620159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:21.632620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-01032b97-ec10-4f40-8faa-95ecee97b5ae None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:21.635556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-01032b97-ec10-4f40-8faa-95ecee97b5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:21.734813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978a62c-0646-4f2c-a324-ffbf8d7f7a8e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:21.741457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:21.742461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:21.742767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:21.743018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] PATH_INFO: `/v3/users/bf836b07e2564a2e8650e6319ad0feed` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:21.745160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf836b07e2564a2e8650e6319ad0feed)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:21.747472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:21.788319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1118/2215] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:21.797309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc6f2-eac2-4f85-96bd-88061b7fb490 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:21.797602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc6f2-eac2-4f85-96bd-88061b7fb490 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:21.797889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc6f2-eac2-4f85-96bd-88061b7fb490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:21.824208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf836b07e2564a2e8650e6319ad0feed'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.824988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-75ebc6f2-eac2-4f85-96bd-88061b7fb490 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:21.827035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-75ebc6f2-eac2-4f85-96bd-88061b7fb490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:21.839328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf836b07e2564a2e8650e6319ad0feed'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.841103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1119/2216] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:21.845249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebc6f2-eac2-4f85-96bd-88061b7fb490 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:21.856863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebc6f2-eac2-4f85-96bd-88061b7fb490 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:21.865274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:21.866359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:21.867020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf836b07e2564a2e8650e6319ad0feed'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.867795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf836b07e2564a2e8650e6319ad0feed'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.869572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:21.869873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:21.870633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:27:21.872504 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:21.876811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:21.888312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e294d2ac-d4da-4d84-b77a-7d33600cc7a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf836b07e2564a2e8650e6319ad0feed'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:21.896119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1098/2217] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:21 2022] DELETE /identity/v3/users/bf836b07e2564a2e8650e6319ad0feed => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:21.901132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6f80f-115c-43af-af8d-6c671cf7540d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:21.901656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6f80f-115c-43af-af8d-6c671cf7540d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:21.901854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6f80f-115c-43af-af8d-6c671cf7540d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:21.903413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1120/2218] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:27:21.906261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:21.907977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:21.921635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-09a6f80f-115c-43af-af8d-6c671cf7540d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:21.921841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9bfa4-2652-4299-a6e9-d44eb14d41f8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:21.922916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-09a6f80f-115c-43af-af8d-6c671cf7540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:22.003918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c10ecf-17cd-4a56-999d-9f04140e1a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.004964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c10ecf-17cd-4a56-999d-9f04140e1a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.005326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c10ecf-17cd-4a56-999d-9f04140e1a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.005699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c10ecf-17cd-4a56-999d-9f04140e1a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.006438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1099/2219] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:22.006914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c10ecf-17cd-4a56-999d-9f04140e1a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.009005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c10ecf-17cd-4a56-999d-9f04140e1a08 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.018493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6f80f-115c-43af-af8d-6c671cf7540d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:22.020077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-09a6f80f-115c-43af-af8d-6c671cf7540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:22.041792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:22.084063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1121/2220] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:21 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:27:22.085917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c10ecf-17cd-4a56-999d-9f04140e1a08 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:22.107488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6f80f-115c-43af-af8d-6c671cf7540d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:22.125958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:22.153665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c10ecf-17cd-4a56-999d-9f04140e1a08 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:22.159893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8b535-fb81-4a91-ab0e-73b38697e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.160976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8b535-fb81-4a91-ab0e-73b38697e8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.161169 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8b535-fb81-4a91-ab0e-73b38697e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.161268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4a062-0672-4007-94a7-c88fbbbe4617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.161443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8b535-fb81-4a91-ab0e-73b38697e8b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.161848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4a062-0672-4007-94a7-c88fbbbe4617 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.162130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4a062-0672-4007-94a7-c88fbbbe4617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.162328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8b535-fb81-4a91-ab0e-73b38697e8b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.162425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4a062-0672-4007-94a7-c88fbbbe4617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.165655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8b535-fb81-4a91-ab0e-73b38697e8b2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.174129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4a062-0672-4007-94a7-c88fbbbe4617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.176137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4a062-0672-4007-94a7-c88fbbbe4617 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.188950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1122/2221] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:22 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:22.192920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8b535-fb81-4a91-ab0e-73b38697e8b2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:22.194340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1100/2222] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:22.203999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8b535-fb81-4a91-ab0e-73b38697e8b2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:22.208759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232697a-3a98-4e4e-a974-b997226d6516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.209421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232697a-3a98-4e4e-a974-b997226d6516 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.209710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232697a-3a98-4e4e-a974-b997226d6516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.209989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232697a-3a98-4e4e-a974-b997226d6516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.210726 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232697a-3a98-4e4e-a974-b997226d6516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.212544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232697a-3a98-4e4e-a974-b997226d6516 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.227324 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1123/2223] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:27:22.228817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4a062-0672-4007-94a7-c88fbbbe4617 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:22.241807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4a062-0672-4007-94a7-c88fbbbe4617 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:22.249953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7254d-4528-483c-ad14-e16381000415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.250697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7254d-4528-483c-ad14-e16381000415 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.251008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7254d-4528-483c-ad14-e16381000415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.251455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7254d-4528-483c-ad14-e16381000415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.252247 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7254d-4528-483c-ad14-e16381000415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.254015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7254d-4528-483c-ad14-e16381000415 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.268149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1101/2224] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:22.272719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232697a-3a98-4e4e-a974-b997226d6516 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:22.286235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232697a-3a98-4e4e-a974-b997226d6516 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:22.291013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.291930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.292217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.292500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] PATH_INFO: `/v3/projects/9afae5d44c6f422bb10f03f930898d48/users/c382adf7d26e428c97d5610b320f31ef/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.316979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9afae5d44c6f422bb10f03f930898d48, user_id=c382adf7d26e428c97d5610b320f31ef, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.319788 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.338177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1124/2225] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:22 2022] PUT /identity/v3/projects/9afae5d44c6f422bb10f03f930898d48/users/c382adf7d26e428c97d5610b320f31ef/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:22.350259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8728-2dc9-44fc-b650-76f45399ce84 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.350548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8728-2dc9-44fc-b650-76f45399ce84 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.350827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8728-2dc9-44fc-b650-76f45399ce84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.379609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fd3c8728-2dc9-44fc-b650-76f45399ce84 None None] MFA Rules not processed for user `c382adf7d26e428c97d5610b320f31ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:22.380842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fd3c8728-2dc9-44fc-b650-76f45399ce84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:22.465382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1102/2226] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:22.477500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:22.479758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:22.502031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c8728-2dc9-44fc-b650-76f45399ce84 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:22.503491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fd3c8728-2dc9-44fc-b650-76f45399ce84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:22.550785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60004a1-6552-4d0f-bab1-4fdea7ea822a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:22.554754 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fa6b5-1a49-420e-800d-7ed94d511088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.555489 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fa6b5-1a49-420e-800d-7ed94d511088 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.555773 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fa6b5-1a49-420e-800d-7ed94d511088 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.556052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fa6b5-1a49-420e-800d-7ed94d511088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.566233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fa6b5-1a49-420e-800d-7ed94d511088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.567815 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fa6b5-1a49-420e-800d-7ed94d511088 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.577638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1125/2227] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:22.579250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:22.586410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c8728-2dc9-44fc-b650-76f45399ce84 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:22.591321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.592259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.592625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.592978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] PATH_INFO: `/v3/projects/aacfe2d2bf374fcfb793d0278a9c8a81` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.594212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=aacfe2d2bf374fcfb793d0278a9c8a81)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.596226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.645497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aacfe2d2bf374fcfb793d0278a9c8a81', 'request_id': 'req-1a7006ee-1a56-4f83-ab76-6112948e1eac'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.656124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aacfe2d2bf374fcfb793d0278a9c8a81', 'request_id': 'req-1a7006ee-1a56-4f83-ab76-6112948e1eac'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.664157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aacfe2d2bf374fcfb793d0278a9c8a81', 'request_id': 'req-1a7006ee-1a56-4f83-ab76-6112948e1eac'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.681511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] The token cache is being invalidate because project aacfe2d2bf374fcfb793d0278a9c8a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:22.681815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.682724 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.684624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1103/2228] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:22 2022] DELETE /identity/v3/projects/aacfe2d2bf374fcfb793d0278a9c8a81 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:22.690018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fa6b5-1a49-420e-800d-7ed94d511088 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:22.707931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fa6b5-1a49-420e-800d-7ed94d511088 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:22.778932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:22.779872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:22.780161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:22.780439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] PATH_INFO: `/v3/users/249f83f112b14f5f9168d45a4389cd18` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:22.781649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=249f83f112b14f5f9168d45a4389cd18)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:22.783733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:22.841316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:22.855743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '249f83f112b14f5f9168d45a4389cd18'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.856865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '249f83f112b14f5f9168d45a4389cd18'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.867457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '249f83f112b14f5f9168d45a4389cd18'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.892452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '249f83f112b14f5f9168d45a4389cd18'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.901286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '249f83f112b14f5f9168d45a4389cd18'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:22.933178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1126/2229] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:22 2022] DELETE /identity/v3/users/249f83f112b14f5f9168d45a4389cd18 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:23.451569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:23.453044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:23.465977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7006ee-1a56-4f83-ab76-6112948e1eac admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:23.544324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:23.546004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:23.546382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:23.546759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] PATH_INFO: `/v3/projects/c61b54aae8794b6e8af417feacdc6fbb` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:23.548170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61b54aae8794b6e8af417feacdc6fbb)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:23.550141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:23.604838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61b54aae8794b6e8af417feacdc6fbb', 'request_id': 'req-5926b385-ff96-4d68-b012-f27f49d1dd36'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.615849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61b54aae8794b6e8af417feacdc6fbb', 'request_id': 'req-5926b385-ff96-4d68-b012-f27f49d1dd36'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.627049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61b54aae8794b6e8af417feacdc6fbb', 'request_id': 'req-5926b385-ff96-4d68-b012-f27f49d1dd36'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.644204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] The token cache is being invalidate because project c61b54aae8794b6e8af417feacdc6fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:23.644497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.645167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.646971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1104/2230] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:23 2022] DELETE /identity/v3/projects/c61b54aae8794b6e8af417feacdc6fbb => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:23.650867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:23.652342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:23.668720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc40993-be6f-4e32-964a-840cc28c8e7d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:23.733636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:23.734530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:23.734812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:23.735087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] PATH_INFO: `/v3/users/326f9726bfd84ba180b7a80ee20b32ad` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:23.736485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326f9726bfd84ba180b7a80ee20b32ad)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:23.738339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:23.805962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326f9726bfd84ba180b7a80ee20b32ad'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.806636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326f9726bfd84ba180b7a80ee20b32ad'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.814054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326f9726bfd84ba180b7a80ee20b32ad'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.827459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326f9726bfd84ba180b7a80ee20b32ad'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.828180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdf086b7-44ac-4316-bc2c-7969b8316611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326f9726bfd84ba180b7a80ee20b32ad'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:23.847938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1127/2231] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:23 2022] DELETE /identity/v3/users/326f9726bfd84ba180b7a80ee20b32ad => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:24.198079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:24.199599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:24.224688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926b385-ff96-4d68-b012-f27f49d1dd36 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:24.241526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:24.336908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:24.337803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:24.338092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:24.338371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] PATH_INFO: `/v3/projects/e027048191e141deb3b78f6a49fe031f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:24.339817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e027048191e141deb3b78f6a49fe031f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:24.343444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:24.426796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e027048191e141deb3b78f6a49fe031f', 'request_id': 'req-70b6c66a-193e-47ea-8538-5acdeb14d088'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.438060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e027048191e141deb3b78f6a49fe031f', 'request_id': 'req-70b6c66a-193e-47ea-8538-5acdeb14d088'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.456494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e027048191e141deb3b78f6a49fe031f', 'request_id': 'req-70b6c66a-193e-47ea-8538-5acdeb14d088'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.468936 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] The token cache is being invalidate because project e027048191e141deb3b78f6a49fe031f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:24.469224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.469886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.471712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1105/2232] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:24 2022] DELETE /identity/v3/projects/e027048191e141deb3b78f6a49fe031f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:24.478187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346fb2f-88f9-4c93-bf3c-0eb562942524 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:24.478678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346fb2f-88f9-4c93-bf3c-0eb562942524 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:24.479087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346fb2f-88f9-4c93-bf3c-0eb562942524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:24.491792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:24.493199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:24.502709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3346fb2f-88f9-4c93-bf3c-0eb562942524 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:24.504158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3346fb2f-88f9-4c93-bf3c-0eb562942524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:24.558768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:24.580827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c66a-193e-47ea-8538-5acdeb14d088 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:24.587730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:24.588918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:24.589347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:24.589712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] PATH_INFO: `/v3/users/c634b1b55ee94c4d82990d4e37359b62` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:24.591078 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c634b1b55ee94c4d82990d4e37359b62)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:24.593793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:24.604598 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1128/2233] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:24.612417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87be2-1879-4297-91a2-2611058321b6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:24.612851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87be2-1879-4297-91a2-2611058321b6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:24.613232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87be2-1879-4297-91a2-2611058321b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:24.635986 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6af87be2-1879-4297-91a2-2611058321b6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:24.637263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6af87be2-1879-4297-91a2-2611058321b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:24.646815 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1129/2234] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:24.649451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c634b1b55ee94c4d82990d4e37359b62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.651025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af87be2-1879-4297-91a2-2611058321b6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:24.651509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:24.660510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c634b1b55ee94c4d82990d4e37359b62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.664106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af87be2-1879-4297-91a2-2611058321b6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:24.669502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:24.670227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:24.670519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:24.670781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:24.671556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:27:24.672180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:24.677456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c634b1b55ee94c4d82990d4e37359b62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.678127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:24.678577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c634b1b55ee94c4d82990d4e37359b62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.688383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c634b1b55ee94c4d82990d4e37359b62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:24.698629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1130/2235] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:27:24.702237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1106/2236] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:24 2022] DELETE /identity/v3/users/c634b1b55ee94c4d82990d4e37359b62 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:27:24.705401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:24.706553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:24.731674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8709739-0476-452a-9179-d07cee1bcc4b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:24.809897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:24.810665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:24.811026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:24.811396 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:24.812382 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:24.814940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:24.846713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1131/2237] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:24 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:27:24.850815 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:24.862019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4c47d-ffda-40ae-9cde-64be168c540d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:24.910728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:24.912547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:24.954729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5767d9d-6e1f-4f92-b287-f7b2940965dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:24.955542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5767d9d-6e1f-4f92-b287-f7b2940965dd admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:24.955866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5767d9d-6e1f-4f92-b287-f7b2940965dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:24.956079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5767d9d-6e1f-4f92-b287-f7b2940965dd admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:24.957090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5767d9d-6e1f-4f92-b287-f7b2940965dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:24.959020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5767d9d-6e1f-4f92-b287-f7b2940965dd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:24.984417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1107/2238] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:24 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:24.988530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5767d9d-6e1f-4f92-b287-f7b2940965dd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:24.999810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a614c-dcd5-441d-ad6d-e7499b186376 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:25.003177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5767d9d-6e1f-4f92-b287-f7b2940965dd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:25.009995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50282518-a360-43bd-b98f-7fe3c1a2953b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:25.010702 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282518-a360-43bd-b98f-7fe3c1a2953b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:25.010983 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282518-a360-43bd-b98f-7fe3c1a2953b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:25.011311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282518-a360-43bd-b98f-7fe3c1a2953b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:25.012116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50282518-a360-43bd-b98f-7fe3c1a2953b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:25.014508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50282518-a360-43bd-b98f-7fe3c1a2953b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:25.016388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:25.017819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:25.018195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:25.018545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] PATH_INFO: `/v3/projects/3cba1f063ee7455c8419c1dceba4ac45` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:25.020008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cba1f063ee7455c8419c1dceba4ac45)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:25.023199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:25.026161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1108/2239] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:27:25.030509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50282518-a360-43bd-b98f-7fe3c1a2953b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:25.041436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50282518-a360-43bd-b98f-7fe3c1a2953b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:25.046556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb1892-20ac-4e60-946f-806d0c7e880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:25.047220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1892-20ac-4e60-946f-806d0c7e880a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:25.047533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1892-20ac-4e60-946f-806d0c7e880a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:25.047811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1892-20ac-4e60-946f-806d0c7e880a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:25.048670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb1892-20ac-4e60-946f-806d0c7e880a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:25.050248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb1892-20ac-4e60-946f-806d0c7e880a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:25.063704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1109/2240] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:25.066515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb1892-20ac-4e60-946f-806d0c7e880a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:25.081656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb1892-20ac-4e60-946f-806d0c7e880a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:25.087917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfab75-3114-4e67-b05d-65c395c47e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:25.088938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfab75-3114-4e67-b05d-65c395c47e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:25.089300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfab75-3114-4e67-b05d-65c395c47e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:25.089607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfab75-3114-4e67-b05d-65c395c47e3b admin admin] PATH_INFO: `/v3/projects/33d09d2e02c945a09e79efae8d8ab96c/users/681c8dec79764371badd23608ff4fa5a/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:25.103945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:25.111745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfab75-3114-4e67-b05d-65c395c47e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d09d2e02c945a09e79efae8d8ab96c, user_id=681c8dec79764371badd23608ff4fa5a, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:25.116158 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cba1f063ee7455c8419c1dceba4ac45', 'request_id': 'req-c5f3b034-7af4-4094-a5b7-9fa624a17d68'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.117254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfab75-3114-4e67-b05d-65c395c47e3b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:25.129048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cba1f063ee7455c8419c1dceba4ac45', 'request_id': 'req-c5f3b034-7af4-4094-a5b7-9fa624a17d68'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.137368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cba1f063ee7455c8419c1dceba4ac45', 'request_id': 'req-c5f3b034-7af4-4094-a5b7-9fa624a17d68'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.145379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1110/2241] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:25 2022] PUT /identity/v3/projects/33d09d2e02c945a09e79efae8d8ab96c/users/681c8dec79764371badd23608ff4fa5a/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:25.151688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f7e8-6ead-4159-9fd0-7b75d623b903 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:25.152169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f7e8-6ead-4159-9fd0-7b75d623b903 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:25.152388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f7e8-6ead-4159-9fd0-7b75d623b903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:25.159026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] The token cache is being invalidate because project 3cba1f063ee7455c8419c1dceba4ac45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:25.159430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.160105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.163275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1132/2242] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:24 2022] DELETE /identity/v3/projects/3cba1f063ee7455c8419c1dceba4ac45 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:25.167472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:25.170284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:25.180534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ebd8f7e8-6ead-4159-9fd0-7b75d623b903 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:25.182218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebd8f7e8-6ead-4159-9fd0-7b75d623b903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:25.320792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1111/2243] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:25.329473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8f7e8-6ead-4159-9fd0-7b75d623b903 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:25.331585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3b034-7af4-4094-a5b7-9fa624a17d68 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:25.332049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebd8f7e8-6ead-4159-9fd0-7b75d623b903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:25.346280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:25.347290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:25.347759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:25.348465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] PATH_INFO: `/v3/users/2fc78cd8cfd54a2a96869d3f019ae64d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:25.350428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc78cd8cfd54a2a96869d3f019ae64d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:25.354027 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:25.361094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:25.429816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8f7e8-6ead-4159-9fd0-7b75d623b903 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:25.434376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:25.435071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc78cd8cfd54a2a96869d3f019ae64d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.435199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:25.435523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:25.436524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:25.437226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc78cd8cfd54a2a96869d3f019ae64d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.438750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:25.445463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc78cd8cfd54a2a96869d3f019ae64d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.456668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:25.459280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:25.463464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc78cd8cfd54a2a96869d3f019ae64d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.468888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a9ca0105-a487-4361-b1f0-891c72c863eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc78cd8cfd54a2a96869d3f019ae64d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.473051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1112/2244] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:25.497740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1133/2245] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:25 2022] DELETE /identity/v3/users/2fc78cd8cfd54a2a96869d3f019ae64d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:25.762440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:25.765584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:25.811158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15058d5-84dd-4832-8702-c335f5d46453 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:25.876122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:25.876996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:25.877438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:25.877885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] PATH_INFO: `/v3/projects/1debbe36ef274dcda31b1fca14faa365` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:25.879208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1debbe36ef274dcda31b1fca14faa365)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:25.881443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:25.948069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1debbe36ef274dcda31b1fca14faa365', 'request_id': 'req-127b3f02-d73a-44eb-aa65-a716f6b27ac2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.957205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1debbe36ef274dcda31b1fca14faa365', 'request_id': 'req-127b3f02-d73a-44eb-aa65-a716f6b27ac2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.963716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1debbe36ef274dcda31b1fca14faa365', 'request_id': 'req-127b3f02-d73a-44eb-aa65-a716f6b27ac2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.975816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] The token cache is being invalidate because project 1debbe36ef274dcda31b1fca14faa365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:25.976107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.976905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:25.978693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1113/2246] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:25 2022] DELETE /identity/v3/projects/1debbe36ef274dcda31b1fca14faa365 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:29.911666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14152b7-23d9-4b38-bc0a-cef743a13328 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:29.912130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14152b7-23d9-4b38-bc0a-cef743a13328 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:29.912219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14152b7-23d9-4b38-bc0a-cef743a13328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:29.959001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b14152b7-23d9-4b38-bc0a-cef743a13328 None None] MFA Rules not processed for user `681c8dec79764371badd23608ff4fa5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:29.960441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b14152b7-23d9-4b38-bc0a-cef743a13328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:29.992797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:30.059095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1134/2247] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:30.064765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:30.066245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:30.125274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:30.203102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b3f02-d73a-44eb-aa65-a716f6b27ac2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:30.207897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b682d-b49b-4b80-92a3-32c452e92016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:30.208743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b682d-b49b-4b80-92a3-32c452e92016 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:30.209035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b682d-b49b-4b80-92a3-32c452e92016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:30.209310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b682d-b49b-4b80-92a3-32c452e92016 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:30.210049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b682d-b49b-4b80-92a3-32c452e92016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:30.212016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b682d-b49b-4b80-92a3-32c452e92016 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:30.244345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1114/2248] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:30 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:30.247635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14152b7-23d9-4b38-bc0a-cef743a13328 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:30.257763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14152b7-23d9-4b38-bc0a-cef743a13328 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:30.314412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5cb4d-bf14-4bef-8f20-f989d3263457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:30.315333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5cb4d-bf14-4bef-8f20-f989d3263457 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:30.315649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5cb4d-bf14-4bef-8f20-f989d3263457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:30.315925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5cb4d-bf14-4bef-8f20-f989d3263457 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:30.316628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5cb4d-bf14-4bef-8f20-f989d3263457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:30.318597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5cb4d-bf14-4bef-8f20-f989d3263457 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:30.339721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1135/2249] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:30 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:30.346190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b682d-b49b-4b80-92a3-32c452e92016 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:30.363568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b682d-b49b-4b80-92a3-32c452e92016 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:30.369689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228fe40-d1df-4409-a6d4-e720a1cc2084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:30.370420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228fe40-d1df-4409-a6d4-e720a1cc2084 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:30.370761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228fe40-d1df-4409-a6d4-e720a1cc2084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:30.371141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228fe40-d1df-4409-a6d4-e720a1cc2084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:30.372310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228fe40-d1df-4409-a6d4-e720a1cc2084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:30.374119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228fe40-d1df-4409-a6d4-e720a1cc2084 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:30.384942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1115/2250] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:30.389521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5cb4d-bf14-4bef-8f20-f989d3263457 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:30.405920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5cb4d-bf14-4bef-8f20-f989d3263457 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:30.411426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba978-5ac2-424b-b27f-beadb777ac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:30.412328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba978-5ac2-424b-b27f-beadb777ac7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:30.412607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba978-5ac2-424b-b27f-beadb777ac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:30.412877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba978-5ac2-424b-b27f-beadb777ac7a admin admin] PATH_INFO: `/v3/projects/9e10e1fb9f6f425cb76033446d29aa8b/users/11639489d3c746068a47376743df065b/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:30.429677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba978-5ac2-424b-b27f-beadb777ac7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e10e1fb9f6f425cb76033446d29aa8b, user_id=11639489d3c746068a47376743df065b, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:30.432603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba978-5ac2-424b-b27f-beadb777ac7a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:30.448322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1136/2251] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:30 2022] PUT /identity/v3/projects/9e10e1fb9f6f425cb76033446d29aa8b/users/11639489d3c746068a47376743df065b/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:30.453301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228fe40-d1df-4409-a6d4-e720a1cc2084 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:30.466628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228fe40-d1df-4409-a6d4-e720a1cc2084 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:30.551175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d1a92-edbb-49e8-88dc-0610bc435715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:30.551892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d1a92-edbb-49e8-88dc-0610bc435715 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:30.552175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d1a92-edbb-49e8-88dc-0610bc435715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:30.552451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d1a92-edbb-49e8-88dc-0610bc435715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:30.553177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d1a92-edbb-49e8-88dc-0610bc435715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:30.554813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d1a92-edbb-49e8-88dc-0610bc435715 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:30.565005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1116/2252] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:27:30.568995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba978-5ac2-424b-b27f-beadb777ac7a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:30.579629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba978-5ac2-424b-b27f-beadb777ac7a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:30.584106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:30.584973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:30.585251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:30.585525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] PATH_INFO: `/v3/projects/9e10e1fb9f6f425cb76033446d29aa8b/users/11639489d3c746068a47376743df065b/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:30.587668 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e10e1fb9f6f425cb76033446d29aa8b, user_id=11639489d3c746068a47376743df065b, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:30.590369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:30.601994 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1137/2253] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:30 2022] PUT /identity/v3/projects/9e10e1fb9f6f425cb76033446d29aa8b/users/11639489d3c746068a47376743df065b/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:36.564668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357dddc-d3dc-4d92-923d-1005aba9526d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:36.565535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357dddc-d3dc-4d92-923d-1005aba9526d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:36.565973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357dddc-d3dc-4d92-923d-1005aba9526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:36.604137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2357dddc-d3dc-4d92-923d-1005aba9526d None None] MFA Rules not processed for user `11639489d3c746068a47376743df065b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:36.605770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2357dddc-d3dc-4d92-923d-1005aba9526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:36.682759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:36.713643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1117/2254] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:36.728622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:36.730993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:36.836143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:36.847757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e22a20-2793-47ef-a64f-cb4618fba730 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:36.851752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:36.852490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:36.852830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:36.852939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:36.944704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:36.947138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:36.965041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1138/2255] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:37.237411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3d0b-ac6f-4a51-aa3b-bfffaa84e3e9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:37.237708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3d0b-ac6f-4a51-aa3b-bfffaa84e3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:37.237997 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3d0b-ac6f-4a51-aa3b-bfffaa84e3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:37.292142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1a8b3d0b-ac6f-4a51-aa3b-bfffaa84e3e9 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:37.293406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1a8b3d0b-ac6f-4a51-aa3b-bfffaa84e3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:37.454003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1118/2256] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:27:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:37.511617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:37.518134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:37.605397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6b4e2-6e42-4da5-9f26-20a09ec99aff service glance] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:37.613231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192c469-fec1-4551-a7bb-6642ad1bff99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:37.613907 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192c469-fec1-4551-a7bb-6642ad1bff99 service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:37.614184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192c469-fec1-4551-a7bb-6642ad1bff99 service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:37.614457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192c469-fec1-4551-a7bb-6642ad1bff99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:37.625603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192c469-fec1-4551-a7bb-6642ad1bff99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:37.627289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192c469-fec1-4551-a7bb-6642ad1bff99 service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:37.636271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1139/2257] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:27:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:27:38.482962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3d0b-ac6f-4a51-aa3b-bfffaa84e3e9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:38.485045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1a8b3d0b-ac6f-4a51-aa3b-bfffaa84e3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:38.626767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3d0b-ac6f-4a51-aa3b-bfffaa84e3e9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:38.633640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:38.634826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:38.635278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:38.635769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] PATH_INFO: `/v3/users/c382adf7d26e428c97d5610b320f31ef` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:38.638070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c382adf7d26e428c97d5610b320f31ef)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:38.641205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:38.733868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c382adf7d26e428c97d5610b320f31ef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:38.752541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c382adf7d26e428c97d5610b320f31ef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:38.783696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c382adf7d26e428c97d5610b320f31ef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:38.784440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c382adf7d26e428c97d5610b320f31ef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:38.800786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c382adf7d26e428c97d5610b320f31ef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:38.813140 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f95c98-f5c6-428d-83aa-84ba65fc0888 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:38.813634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f95c98-f5c6-428d-83aa-84ba65fc0888 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:38.813634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f95c98-f5c6-428d-83aa-84ba65fc0888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:38.815152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1119/2258] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:38 2022] DELETE /identity/v3/users/c382adf7d26e428c97d5610b320f31ef => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:38.866810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-85f95c98-f5c6-428d-83aa-84ba65fc0888 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:38.874397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-85f95c98-f5c6-428d-83aa-84ba65fc0888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:39.081760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1140/2259] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:39.091947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:39.094124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:39.114374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:39.209764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb72e-bd55-4b48-98d0-32a40fc0e389 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:39.214152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324076d2-bf1a-4862-9ace-d691c56652aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:39.214995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324076d2-bf1a-4862-9ace-d691c56652aa service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:39.215285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324076d2-bf1a-4862-9ace-d691c56652aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:39.215597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324076d2-bf1a-4862-9ace-d691c56652aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:39.227244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324076d2-bf1a-4862-9ace-d691c56652aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:39.228904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324076d2-bf1a-4862-9ace-d691c56652aa service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:39.254518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1120/2260] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:39.666545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f95c98-f5c6-428d-83aa-84ba65fc0888 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:39.671723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-85f95c98-f5c6-428d-83aa-84ba65fc0888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:39.696127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f95c98-f5c6-428d-83aa-84ba65fc0888 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:39.726500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:39.727553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:39.727941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:39.728327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] PATH_INFO: `/v3/projects/9afae5d44c6f422bb10f03f930898d48` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:39.732539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9afae5d44c6f422bb10f03f930898d48)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:39.734658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:39.763198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:39.819932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9afae5d44c6f422bb10f03f930898d48', 'request_id': 'req-df5760ea-5969-4309-8379-098d33c40062'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:39.832260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9afae5d44c6f422bb10f03f930898d48', 'request_id': 'req-df5760ea-5969-4309-8379-098d33c40062'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:39.854192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9afae5d44c6f422bb10f03f930898d48', 'request_id': 'req-df5760ea-5969-4309-8379-098d33c40062'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:39.876099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] The token cache is being invalidate because project 9afae5d44c6f422bb10f03f930898d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:39.876385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:39.878198 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df5760ea-5969-4309-8379-098d33c40062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:39.880201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1141/2261] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:39 2022] DELETE /identity/v3/projects/9afae5d44c6f422bb10f03f930898d48 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:39.884959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3cd9e-c847-4fc9-9188-f0cab76f3481 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:39.885461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3cd9e-c847-4fc9-9188-f0cab76f3481 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:39.885461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3cd9e-c847-4fc9-9188-f0cab76f3481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:39.944232 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e1d3cd9e-c847-4fc9-9188-f0cab76f3481 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:39.946068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e1d3cd9e-c847-4fc9-9188-f0cab76f3481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:40.039134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1121/2262] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:40.041385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666e4d-3f47-4417-842b-1d038a339f11 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:40.041913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666e4d-3f47-4417-842b-1d038a339f11 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:40.042196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666e4d-3f47-4417-842b-1d038a339f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:40.085981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-59666e4d-3f47-4417-842b-1d038a339f11 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:40.088239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-59666e4d-3f47-4417-842b-1d038a339f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:40.114528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1142/2263] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:40.118209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3cd9e-c847-4fc9-9188-f0cab76f3481 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:40.130030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3cd9e-c847-4fc9-9188-f0cab76f3481 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:40.136939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:40.139176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:40.139495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:40.139859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:40.140632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:27:40.140945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:40.147215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:40.166248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1122/2264] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:27:40.167044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59666e4d-3f47-4417-842b-1d038a339f11 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:40.179557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59666e4d-3f47-4417-842b-1d038a339f11 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:40.193031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dfde9-60cc-4537-9981-277ba50c7896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:40.193714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dfde9-60cc-4537-9981-277ba50c7896 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:40.194076 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dfde9-60cc-4537-9981-277ba50c7896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:40.197353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dfde9-60cc-4537-9981-277ba50c7896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:40.198110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dfde9-60cc-4537-9981-277ba50c7896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:40.200083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dfde9-60cc-4537-9981-277ba50c7896 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:40.250871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1143/2265] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:40 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:40.255657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:40.277042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7f6fe-f143-4eeb-bd97-9e85acca0a0e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:40.386092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22ec1-dcd8-4c3f-b1c4-a0aa496412a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:40.386824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22ec1-dcd8-4c3f-b1c4-a0aa496412a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:40.387156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22ec1-dcd8-4c3f-b1c4-a0aa496412a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:40.387414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22ec1-dcd8-4c3f-b1c4-a0aa496412a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:40.390752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d22ec1-dcd8-4c3f-b1c4-a0aa496412a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:40.394168 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d22ec1-dcd8-4c3f-b1c4-a0aa496412a9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:40.408770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:40.448776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1123/2266] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:40 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:40.451842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dfde9-60cc-4537-9981-277ba50c7896 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:40.468041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dfde9-60cc-4537-9981-277ba50c7896 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:40.474830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d580e-a911-47b9-af70-719c124be7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:40.475560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d580e-a911-47b9-af70-719c124be7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:40.475840 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d580e-a911-47b9-af70-719c124be7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:40.476118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d580e-a911-47b9-af70-719c124be7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:40.479385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d580e-a911-47b9-af70-719c124be7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:40.481008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d580e-a911-47b9-af70-719c124be7a3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:40.511390 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1144/2267] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:40.512595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22ec1-dcd8-4c3f-b1c4-a0aa496412a9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:40.527328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22ec1-dcd8-4c3f-b1c4-a0aa496412a9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:40.539872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393558d8-2b96-4546-bf00-3334d43b01b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:40.542081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393558d8-2b96-4546-bf00-3334d43b01b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:40.542695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393558d8-2b96-4546-bf00-3334d43b01b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:40.543469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393558d8-2b96-4546-bf00-3334d43b01b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:40.545395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393558d8-2b96-4546-bf00-3334d43b01b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:40.547669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393558d8-2b96-4546-bf00-3334d43b01b2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:40.567904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1124/2268] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:40.572284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d580e-a911-47b9-af70-719c124be7a3 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:40.585703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d580e-a911-47b9-af70-719c124be7a3 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:40.600613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:40.601521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:40.601803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:40.602078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] PATH_INFO: `/v3/projects/ac9720507b21405693b732e3db51bec9/users/3c3a209d6b074da6a6baaa2223634110/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:40.625378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9720507b21405693b732e3db51bec9, user_id=3c3a209d6b074da6a6baaa2223634110, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:40.628116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:40.735377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1145/2269] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:40 2022] PUT /identity/v3/projects/ac9720507b21405693b732e3db51bec9/users/3c3a209d6b074da6a6baaa2223634110/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:40.743951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986856-6560-4a6f-a0d1-243d48b3bfe8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:40.744393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986856-6560-4a6f-a0d1-243d48b3bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:40.745051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986856-6560-4a6f-a0d1-243d48b3bfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:40.819230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-63986856-6560-4a6f-a0d1-243d48b3bfe8 None None] MFA Rules not processed for user `3c3a209d6b074da6a6baaa2223634110`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:40.824605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-63986856-6560-4a6f-a0d1-243d48b3bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:40.957356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1125/2270] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:40.970626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:40.972644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:41.052548 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116552-7300-4039-9122-793aa49c9297 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:41.062770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1494-8471-4733-946b-3f36b32d94ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:41.064172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1494-8471-4733-946b-3f36b32d94ba service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:41.064567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1494-8471-4733-946b-3f36b32d94ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:41.065048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1494-8471-4733-946b-3f36b32d94ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:41.085278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a1494-8471-4733-946b-3f36b32d94ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:41.086903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a1494-8471-4733-946b-3f36b32d94ba service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:41.108408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1146/2271] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:45.681704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63986856-6560-4a6f-a0d1-243d48b3bfe8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:45.760742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63986856-6560-4a6f-a0d1-243d48b3bfe8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:45.766113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:45.767147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:45.767557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:45.767932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] PATH_INFO: `/v3/users/3c3a209d6b074da6a6baaa2223634110` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:45.769351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3a209d6b074da6a6baaa2223634110)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:45.771381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:45.845381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3a209d6b074da6a6baaa2223634110'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:45.856149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3a209d6b074da6a6baaa2223634110'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:45.876090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3a209d6b074da6a6baaa2223634110'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:45.876916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3a209d6b074da6a6baaa2223634110'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:45.881988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:45.892065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3a209d6b074da6a6baaa2223634110'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:45.908134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1126/2272] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:27:45 2022] DELETE /identity/v3/users/3c3a209d6b074da6a6baaa2223634110 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:45.909637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03ef8f-60d4-44c0-a626-d8507c9a59b8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:45.910461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03ef8f-60d4-44c0-a626-d8507c9a59b8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:45.910899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03ef8f-60d4-44c0-a626-d8507c9a59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:45.940012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1c03ef8f-60d4-44c0-a626-d8507c9a59b8 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:45.942814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1c03ef8f-60d4-44c0-a626-d8507c9a59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:46.041128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:46.075017 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1147/2273] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:46.085199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:46.086640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:46.156080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed6240-165e-478c-a01b-a5c67ec3bd90 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:46.163193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d9078-c892-44ab-9d4a-13e614746d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:46.167914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d9078-c892-44ab-9d4a-13e614746d32 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:46.170455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d9078-c892-44ab-9d4a-13e614746d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:46.170914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d9078-c892-44ab-9d4a-13e614746d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:46.207467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d9078-c892-44ab-9d4a-13e614746d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:46.209136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d9078-c892-44ab-9d4a-13e614746d32 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:46.236959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1127/2274] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:46.711145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03ef8f-60d4-44c0-a626-d8507c9a59b8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:46.713487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1c03ef8f-60d4-44c0-a626-d8507c9a59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:46.729333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03ef8f-60d4-44c0-a626-d8507c9a59b8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:46.741107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:46.742129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:46.742486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:46.742834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] PATH_INFO: `/v3/projects/ac9720507b21405693b732e3db51bec9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:46.744305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9720507b21405693b732e3db51bec9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:46.746385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:46.815152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9720507b21405693b732e3db51bec9', 'request_id': 'req-bbd344e0-1e43-41e7-8447-0851bb908964'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:46.822097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9720507b21405693b732e3db51bec9', 'request_id': 'req-bbd344e0-1e43-41e7-8447-0851bb908964'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:46.831326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9720507b21405693b732e3db51bec9', 'request_id': 'req-bbd344e0-1e43-41e7-8447-0851bb908964'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:46.847300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] The token cache is being invalidate because project ac9720507b21405693b732e3db51bec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:27:46.847638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:46.848269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bbd344e0-1e43-41e7-8447-0851bb908964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:27:46.850029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1148/2275] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:27:46 2022] DELETE /identity/v3/projects/ac9720507b21405693b732e3db51bec9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:46.855011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56b6e-4db4-4a54-9bb4-cd13bf3bcb10 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:46.856094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56b6e-4db4-4a54-9bb4-cd13bf3bcb10 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:46.856584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56b6e-4db4-4a54-9bb4-cd13bf3bcb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:46.893398 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0df56b6e-4db4-4a54-9bb4-cd13bf3bcb10 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:46.894732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0df56b6e-4db4-4a54-9bb4-cd13bf3bcb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:46.972459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1128/2276] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:46.974156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f4e6-b4f2-4fb4-9c3d-97d54bb9c2a1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:46.974616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f4e6-b4f2-4fb4-9c3d-97d54bb9c2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:46.974697 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f4e6-b4f2-4fb4-9c3d-97d54bb9c2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.000203 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-10f6f4e6-b4f2-4fb4-9c3d-97d54bb9c2a1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:47.001847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-10f6f4e6-b4f2-4fb4-9c3d-97d54bb9c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:47.012893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1149/2277] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:47.017055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df56b6e-4db4-4a54-9bb4-cd13bf3bcb10 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:47.037330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df56b6e-4db4-4a54-9bb4-cd13bf3bcb10 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:47.058801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:47.059581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:47.059866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:47.060219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.061487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:27:47.061941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:47.063622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:47.083274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1129/2278] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:27:47.089214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6f4e6-b4f2-4fb4-9c3d-97d54bb9c2a1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:47.100881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6f4e6-b4f2-4fb4-9c3d-97d54bb9c2a1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:47.107528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fba814-7393-4848-960c-7d30bfa9df1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:47.108202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fba814-7393-4848-960c-7d30bfa9df1a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:47.108473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fba814-7393-4848-960c-7d30bfa9df1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:47.108744 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fba814-7393-4848-960c-7d30bfa9df1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.109851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fba814-7393-4848-960c-7d30bfa9df1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:47.111797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fba814-7393-4848-960c-7d30bfa9df1a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:47.161196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:47.161865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1150/2279] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:27:47 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:27:47.175960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd77e52-1965-4815-90cd-529d0bc24ed3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:47.257705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f5452-7b03-42f4-86ba-3e1d40018b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:47.258549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5452-7b03-42f4-86ba-3e1d40018b93 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:47.258735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5452-7b03-42f4-86ba-3e1d40018b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:47.259013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5452-7b03-42f4-86ba-3e1d40018b93 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.259830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f5452-7b03-42f4-86ba-3e1d40018b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:47.271208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f5452-7b03-42f4-86ba-3e1d40018b93 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:47.287983 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1130/2280] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:27:47 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:27:47.292159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fba814-7393-4848-960c-7d30bfa9df1a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:47.305568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fba814-7393-4848-960c-7d30bfa9df1a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:47.320246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaef9a-311d-4d48-8618-221aac14cf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:47.321224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaef9a-311d-4d48-8618-221aac14cf14 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:47.321498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaef9a-311d-4d48-8618-221aac14cf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:47.321763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaef9a-311d-4d48-8618-221aac14cf14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.322804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aaef9a-311d-4d48-8618-221aac14cf14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:47.324420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aaef9a-311d-4d48-8618-221aac14cf14 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:47.338681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1151/2281] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:27:47.343130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f5452-7b03-42f4-86ba-3e1d40018b93 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:47.348352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:47.354470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f5452-7b03-42f4-86ba-3e1d40018b93 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:47.366141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6194b3-ad76-476d-9859-89b4d43bef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:47.366934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6194b3-ad76-476d-9859-89b4d43bef18 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:47.367282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6194b3-ad76-476d-9859-89b4d43bef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:47.367656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6194b3-ad76-476d-9859-89b4d43bef18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.368467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6194b3-ad76-476d-9859-89b4d43bef18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:47.370181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6194b3-ad76-476d-9859-89b4d43bef18 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:47.383574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1131/2282] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:27:47.388026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaef9a-311d-4d48-8618-221aac14cf14 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:47.400748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaef9a-311d-4d48-8618-221aac14cf14 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:47.408455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:47.409413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:47.409691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:47.409965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] PATH_INFO: `/v3/projects/9065845ed3d0457c9ab0889760958a32/users/14273a09857f431bb7841b321b913c00/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.426752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9065845ed3d0457c9ab0889760958a32, user_id=14273a09857f431bb7841b321b913c00, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:47.430561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:47.446561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1152/2283] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:27:47 2022] PUT /identity/v3/projects/9065845ed3d0457c9ab0889760958a32/users/14273a09857f431bb7841b321b913c00/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:27:47.452650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab82a2-d1f2-4976-b53e-22acfdd605f6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:47.452944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab82a2-d1f2-4976-b53e-22acfdd605f6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:47.453224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab82a2-d1f2-4976-b53e-22acfdd605f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.474556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6bab82a2-d1f2-4976-b53e-22acfdd605f6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:47.475842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6bab82a2-d1f2-4976-b53e-22acfdd605f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:47.542225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1132/2284] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:47.552461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:47.554188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:47.625116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf861e4-bbca-4d4a-83ec-ab894d2bc62d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:47.631877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:47.632562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:47.632848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:47.633114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:47.643694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:47.645141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:47.660457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1153/2285] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:48.684662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003704-d719-4884-bcfd-9af2abb07b5b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:48.684662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003704-d719-4884-bcfd-9af2abb07b5b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:48.684662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003704-d719-4884-bcfd-9af2abb07b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:48.743080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4e003704-d719-4884-bcfd-9af2abb07b5b None None] MFA Rules not processed for user `14273a09857f431bb7841b321b913c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:48.744523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4e003704-d719-4884-bcfd-9af2abb07b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:48.850708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1133/2286] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:48.859175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:48.861412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:48.880322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:48.947460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de4fc3-90cf-4519-9610-0a73237a84bf service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:48.952389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeab33-6290-4aef-a3f9-33f52aad2df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:48.953108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeab33-6290-4aef-a3f9-33f52aad2df9 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:48.953428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeab33-6290-4aef-a3f9-33f52aad2df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:48.953672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeab33-6290-4aef-a3f9-33f52aad2df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:48.965217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeab33-6290-4aef-a3f9-33f52aad2df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:48.968405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeab33-6290-4aef-a3f9-33f52aad2df9 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:48.980547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1154/2287] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:27:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:27:49.533457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bf5ee-9d91-44de-b70a-e9805d41d2a0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:49.533752 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bf5ee-9d91-44de-b70a-e9805d41d2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:49.534057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bf5ee-9d91-44de-b70a-e9805d41d2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:49.556165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7c2bf5ee-9d91-44de-b70a-e9805d41d2a0 None None] MFA Rules not processed for user `681c8dec79764371badd23608ff4fa5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:49.557368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7c2bf5ee-9d91-44de-b70a-e9805d41d2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:49.612539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1134/2288] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:50.422781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13990c9d-1871-4346-b236-8ebc809023a8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:50.423286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13990c9d-1871-4346-b236-8ebc809023a8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:50.423422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13990c9d-1871-4346-b236-8ebc809023a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:50.442383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-13990c9d-1871-4346-b236-8ebc809023a8 None None] MFA Rules not processed for user `14273a09857f431bb7841b321b913c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:50.443778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-13990c9d-1871-4346-b236-8ebc809023a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:50.455391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1155/2289] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:27:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:27:51.901785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df97019-6302-4b53-83b2-680e5f4c95d0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:51.902157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df97019-6302-4b53-83b2-680e5f4c95d0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:51.902341 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df97019-6302-4b53-83b2-680e5f4c95d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:51.947164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3df97019-6302-4b53-83b2-680e5f4c95d0 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:27:51.948964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3df97019-6302-4b53-83b2-680e5f4c95d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:51.966082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:27:52.024220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1135/2290] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:27:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:27:52.039753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13990c9d-1871-4346-b236-8ebc809023a8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:27:52.042186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-13990c9d-1871-4346-b236-8ebc809023a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:27:52.133335 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13990c9d-1871-4346-b236-8ebc809023a8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:27:52.139376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66891fd3-e5de-4abc-8bd0-74cf961452e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:27:52.140036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66891fd3-e5de-4abc-8bd0-74cf961452e0 service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:27:52.140319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66891fd3-e5de-4abc-8bd0-74cf961452e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:27:52.140593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66891fd3-e5de-4abc-8bd0-74cf961452e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:27:52.151202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66891fd3-e5de-4abc-8bd0-74cf961452e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:27:52.152820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66891fd3-e5de-4abc-8bd0-74cf961452e0 service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:27:52.162644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1156/2291] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:27:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:28:02.013059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=17 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:28:02.159166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=57 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:28:20.423980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df97019-6302-4b53-83b2-680e5f4c95d0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:28:20.425548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3df97019-6302-4b53-83b2-680e5f4c95d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:20.439788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df97019-6302-4b53-83b2-680e5f4c95d0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:28:20.444364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:28:20.445238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:28:20.445519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:28:20.445794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] PATH_INFO: `/v3/users/c855c78b08b145fba6590767356578ad` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:28:20.446979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c855c78b08b145fba6590767356578ad)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:28:20.448925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:28:20.495321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c855c78b08b145fba6590767356578ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:20.503879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c855c78b08b145fba6590767356578ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:20.519803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c855c78b08b145fba6590767356578ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:20.519803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c855c78b08b145fba6590767356578ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:20.524444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c855c78b08b145fba6590767356578ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:20.530322 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1136/2292] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:28:20 2022] DELETE /identity/v3/users/c855c78b08b145fba6590767356578ad => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:28:20.535392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb53d58-144c-4ed8-9343-ba44df3211e4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:28:20.535891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb53d58-144c-4ed8-9343-ba44df3211e4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:28:20.535989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb53d58-144c-4ed8-9343-ba44df3211e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:28:20.555328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9cb53d58-144c-4ed8-9343-ba44df3211e4 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:28:20.556541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9cb53d58-144c-4ed8-9343-ba44df3211e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:20.616877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1157/2293] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:28:20.626952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:28:20.628390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:20.684474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c555a-bc77-46bf-b45a-20a73f497249 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:28:20.688801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:28:20.689439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:28:20.689723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:28:20.690000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:28:20.698942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:28:20.700403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:28:20.709898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1137/2294] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:28:20.972525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb53d58-144c-4ed8-9343-ba44df3211e4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:28:20.974048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9cb53d58-144c-4ed8-9343-ba44df3211e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:20.990476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb53d58-144c-4ed8-9343-ba44df3211e4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:28:20.995288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:28:20.996233 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:28:20.996505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:28:20.996782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] PATH_INFO: `/v3/projects/589e1906e2d0471d8f1cf677d3f5d41a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:28:20.998880 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=589e1906e2d0471d8f1cf677d3f5d41a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:28:21.001283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:28:21.061617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589e1906e2d0471d8f1cf677d3f5d41a', 'request_id': 'req-b23c3f88-7181-404a-af75-4417d63414d6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:21.070414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589e1906e2d0471d8f1cf677d3f5d41a', 'request_id': 'req-b23c3f88-7181-404a-af75-4417d63414d6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:21.076496 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589e1906e2d0471d8f1cf677d3f5d41a', 'request_id': 'req-b23c3f88-7181-404a-af75-4417d63414d6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:21.088877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] The token cache is being invalidate because project 589e1906e2d0471d8f1cf677d3f5d41a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:28:21.089252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:21.089952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b23c3f88-7181-404a-af75-4417d63414d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:28:21.091552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1158/2295] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:28:20 2022] DELETE /identity/v3/projects/589e1906e2d0471d8f1cf677d3f5d41a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:28:30.705653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:28:31.085552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:28:41.051245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:28:41.053286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:41.174583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9767c-7ace-458d-83dd-8906b4284326 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:28:41.244673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad570e6c-5f3b-4b20-a910-22537493324a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:28:41.245258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570e6c-5f3b-4b20-a910-22537493324a service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:28:41.245548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570e6c-5f3b-4b20-a910-22537493324a service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:28:41.245826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570e6c-5f3b-4b20-a910-22537493324a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:28:41.247785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ad570e6c-5f3b-4b20-a910-22537493324a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:41.262111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad570e6c-5f3b-4b20-a910-22537493324a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:28:41.264515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad570e6c-5f3b-4b20-a910-22537493324a service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:28:41.279391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1138/2296] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:28:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:28:48.260505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa123ca6-9dd8-4543-b3b3-7e741982e2f6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:28:48.261062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa123ca6-9dd8-4543-b3b3-7e741982e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:28:48.261062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa123ca6-9dd8-4543-b3b3-7e741982e2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:28:48.279895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-aa123ca6-9dd8-4543-b3b3-7e741982e2f6 None None] MFA Rules not processed for user `927f21ef1ca74ec58f85181728933e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:28:48.281090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-aa123ca6-9dd8-4543-b3b3-7e741982e2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:48.340518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1159/2297] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:28:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:28:51.273296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=67 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:28:54.167961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384839a-af7f-49db-a602-cbf52b302f38 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:28:54.168545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384839a-af7f-49db-a602-cbf52b302f38 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:28:54.168727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384839a-af7f-49db-a602-cbf52b302f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:28:54.203953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0384839a-af7f-49db-a602-cbf52b302f38 None None] MFA Rules not processed for user `14273a09857f431bb7841b321b913c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:28:54.205265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0384839a-af7f-49db-a602-cbf52b302f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:54.275814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1139/2298] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:28:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:28:58.333420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=25 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:28:59.103006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a400c8-0309-4164-adba-676aec0f2e1c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:28:59.103302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a400c8-0309-4164-adba-676aec0f2e1c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:28:59.103610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a400c8-0309-4164-adba-676aec0f2e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:28:59.121680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-11a400c8-0309-4164-adba-676aec0f2e1c None None] MFA Rules not processed for user `b6a6d8ab7cc04e14b8f148412974e995`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:28:59.122868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-11a400c8-0309-4164-adba-676aec0f2e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:28:59.179090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1160/2299] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:28:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:29:03.295723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384839a-af7f-49db-a602-cbf52b302f38 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:03.297204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0384839a-af7f-49db-a602-cbf52b302f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:03.359979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384839a-af7f-49db-a602-cbf52b302f38 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:03.363630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:03.364425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:03.364842 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:03.365136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:03.366990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:03.442457 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:03.443959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:03.455067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1140/2300] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:29:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:29:09.171740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=25 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:29:13.449768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=75 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:29:36.038809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961966-621b-4c98-8197-bda6ad53e0aa None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:36.039493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961966-621b-4c98-8197-bda6ad53e0aa None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:36.039493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961966-621b-4c98-8197-bda6ad53e0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:36.094650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8f961966-621b-4c98-8197-bda6ad53e0aa None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:29:36.096098 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8f961966-621b-4c98-8197-bda6ad53e0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:36.188372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1161/2301] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:29:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:29:36.232305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:36.233798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:36.324327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a213ed-ce8b-4605-b8a4-c253516f94ef service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:36.329364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77de32-f71d-4ca8-8b54-7d1eac2805c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:36.329957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77de32-f71d-4ca8-8b54-7d1eac2805c0 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:36.330239 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77de32-f71d-4ca8-8b54-7d1eac2805c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:36.330513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77de32-f71d-4ca8-8b54-7d1eac2805c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:36.338505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77de32-f71d-4ca8-8b54-7d1eac2805c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:36.340130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77de32-f71d-4ca8-8b54-7d1eac2805c0 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:36.348219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1141/2302] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:29:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:29:46.185095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=43 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:29:46.344925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=41 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:29:51.617269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143cc7-0364-4d42-8495-25e5565d2c82 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:51.617793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143cc7-0364-4d42-8495-25e5565d2c82 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:51.617793 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143cc7-0364-4d42-8495-25e5565d2c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:51.635791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e1143cc7-0364-4d42-8495-25e5565d2c82 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:29:51.636977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e1143cc7-0364-4d42-8495-25e5565d2c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:51.644187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1162/2303] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 17:29:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:29:51.656476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77de32-f71d-4ca8-8b54-7d1eac2805c0 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:51.668087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77de32-f71d-4ca8-8b54-7d1eac2805c0 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:51.673242 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f14a3-73c7-4b5b-be29-92ddcb5260fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:51.673920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f14a3-73c7-4b5b-be29-92ddcb5260fe service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:51.674201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f14a3-73c7-4b5b-be29-92ddcb5260fe service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:51.674482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f14a3-73c7-4b5b-be29-92ddcb5260fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:51.683415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f14a3-73c7-4b5b-be29-92ddcb5260fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:51.684976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f14a3-73c7-4b5b-be29-92ddcb5260fe service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:51.693153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1142/2304] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:29:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:29:53.629806 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143cc7-0364-4d42-8495-25e5565d2c82 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:53.631359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e1143cc7-0364-4d42-8495-25e5565d2c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:53.701951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143cc7-0364-4d42-8495-25e5565d2c82 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:53.706537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:53.707491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:53.707818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:53.708092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] PATH_INFO: `/v3/users/14273a09857f431bb7841b321b913c00` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:53.709368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14273a09857f431bb7841b321b913c00)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:53.711565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:53.762922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14273a09857f431bb7841b321b913c00'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:53.763608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14273a09857f431bb7841b321b913c00'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:53.771559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14273a09857f431bb7841b321b913c00'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:53.783415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14273a09857f431bb7841b321b913c00'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:53.784183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4ef88272-2438-49d2-bbfc-64b6c08a8e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14273a09857f431bb7841b321b913c00'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:53.798270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1163/2305] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:29:53 2022] DELETE /identity/v3/users/14273a09857f431bb7841b321b913c00 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:29:53.972942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f14a3-73c7-4b5b-be29-92ddcb5260fe service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:53.984534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f14a3-73c7-4b5b-be29-92ddcb5260fe service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:54.042969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:54.043864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.044144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.044417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] PATH_INFO: `/v3/projects/9065845ed3d0457c9ab0889760958a32` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.045579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9065845ed3d0457c9ab0889760958a32)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:54.047498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:54.093185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9065845ed3d0457c9ab0889760958a32', 'request_id': 'req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:54.101145 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9065845ed3d0457c9ab0889760958a32', 'request_id': 'req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:54.107235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9065845ed3d0457c9ab0889760958a32', 'request_id': 'req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:54.117329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] The token cache is being invalidate because project 9065845ed3d0457c9ab0889760958a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:29:54.117637 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:54.118256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8b0db6cc-12bc-485b-ae1b-aef779bfe67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:29:54.119821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1143/2306] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:29:53 2022] DELETE /identity/v3/projects/9065845ed3d0457c9ab0889760958a32 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:29:54.124821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3439d1-d1f4-47f2-a8f9-bd2ca0b43522 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.125362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3439d1-d1f4-47f2-a8f9-bd2ca0b43522 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.125638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3439d1-d1f4-47f2-a8f9-bd2ca0b43522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.146692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-da3439d1-d1f4-47f2-a8f9-bd2ca0b43522 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:29:54.147918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-da3439d1-d1f4-47f2-a8f9-bd2ca0b43522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:54.210368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1164/2307] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:29:54.218485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002942-26dc-4ab2-81bc-515561dbfadf None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.218771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002942-26dc-4ab2-81bc-515561dbfadf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.219043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002942-26dc-4ab2-81bc-515561dbfadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.238424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-75002942-26dc-4ab2-81bc-515561dbfadf None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:29:54.239719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-75002942-26dc-4ab2-81bc-515561dbfadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:54.247487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1144/2308] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:29:54.252043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3439d1-d1f4-47f2-a8f9-bd2ca0b43522 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:54.262305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3439d1-d1f4-47f2-a8f9-bd2ca0b43522 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:54.266823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:54.267506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.267790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.268064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.268840 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:29:54.269122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:54.270720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:54.279129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:29:54.283252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1165/2309] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:29:54.287774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75002942-26dc-4ab2-81bc-515561dbfadf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:54.296981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75002942-26dc-4ab2-81bc-515561dbfadf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:54.302097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00f9e4-02d4-4e9c-ab2e-e0fa93f5a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:54.302764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00f9e4-02d4-4e9c-ab2e-e0fa93f5a667 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.303044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00f9e4-02d4-4e9c-ab2e-e0fa93f5a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.303315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00f9e4-02d4-4e9c-ab2e-e0fa93f5a667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.304032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00f9e4-02d4-4e9c-ab2e-e0fa93f5a667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:54.305920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00f9e4-02d4-4e9c-ab2e-e0fa93f5a667 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:54.330830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1145/2310] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:29:54 2022] POST /identity/v3/projects => generated 397 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:29:54.334947 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:54.344881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89047911-1c18-4575-90c5-769cedbee783 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:54.402005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5028b-2381-4e58-85d4-d76fa758a6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:54.402676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5028b-2381-4e58-85d4-d76fa758a6ba admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.402962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5028b-2381-4e58-85d4-d76fa758a6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.403244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5028b-2381-4e58-85d4-d76fa758a6ba admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.403985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5028b-2381-4e58-85d4-d76fa758a6ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:54.405919 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5028b-2381-4e58-85d4-d76fa758a6ba admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:54.427376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1166/2311] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:29:54 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:29:54.431333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00f9e4-02d4-4e9c-ab2e-e0fa93f5a667 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:54.444730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00f9e4-02d4-4e9c-ab2e-e0fa93f5a667 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:54.450538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb29b8-f233-4124-9ca3-04466975b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:54.451225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb29b8-f233-4124-9ca3-04466975b452 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.452080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb29b8-f233-4124-9ca3-04466975b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.452469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb29b8-f233-4124-9ca3-04466975b452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.453260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb29b8-f233-4124-9ca3-04466975b452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:54.455281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb29b8-f233-4124-9ca3-04466975b452 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:54.468573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1146/2312] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:29:54.472632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5028b-2381-4e58-85d4-d76fa758a6ba admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:54.483463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5028b-2381-4e58-85d4-d76fa758a6ba admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:54.488173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917a4ff-0c1d-41c3-b7d1-bf60fd5796f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:54.489405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917a4ff-0c1d-41c3-b7d1-bf60fd5796f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.489650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917a4ff-0c1d-41c3-b7d1-bf60fd5796f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.489923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917a4ff-0c1d-41c3-b7d1-bf60fd5796f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.490645 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917a4ff-0c1d-41c3-b7d1-bf60fd5796f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:54.492298 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917a4ff-0c1d-41c3-b7d1-bf60fd5796f0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:54.505092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1167/2313] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:29:54.509768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb29b8-f233-4124-9ca3-04466975b452 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:54.520969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb29b8-f233-4124-9ca3-04466975b452 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:54.528223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:54.529388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.529966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.530323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] PATH_INFO: `/v3/projects/664ab4816a9f4151b81de70e94b49a5e/users/76a7ffd5f76f447fa517cfef267c842a/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.547830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] RBAC: Authorizing `identity:create_grant(project_id=664ab4816a9f4151b81de70e94b49a5e, user_id=76a7ffd5f76f447fa517cfef267c842a, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:54.550514 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:54.571563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1147/2314] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:29:54 2022] PUT /identity/v3/projects/664ab4816a9f4151b81de70e94b49a5e/users/76a7ffd5f76f447fa517cfef267c842a/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:29:54.580590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1c77-1787-4435-acf8-0ec4a2e843df None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.580971 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1c77-1787-4435-acf8-0ec4a2e843df None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.581201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1c77-1787-4435-acf8-0ec4a2e843df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.610181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c54c1c77-1787-4435-acf8-0ec4a2e843df None None] MFA Rules not processed for user `76a7ffd5f76f447fa517cfef267c842a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:29:54.611416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c54c1c77-1787-4435-acf8-0ec4a2e843df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:54.684745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1168/2315] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:29:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:29:54.696440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:29:54.698180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:29:54.721375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:29:54.786492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47047b53-4d17-41d1-8ee8-17a81ede167b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:29:54.790320 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74230418-de74-4ef7-85ae-c70f0b26de8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:29:54.790984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230418-de74-4ef7-85ae-c70f0b26de8a service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:29:54.791273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230418-de74-4ef7-85ae-c70f0b26de8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:29:54.791601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230418-de74-4ef7-85ae-c70f0b26de8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:29:54.800672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230418-de74-4ef7-85ae-c70f0b26de8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:29:54.802655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230418-de74-4ef7-85ae-c70f0b26de8a service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:29:54.811992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1148/2316] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:29:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:30:03.920362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1c77-1787-4435-acf8-0ec4a2e843df None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:03.983129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1c77-1787-4435-acf8-0ec4a2e843df None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:03.987619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:03.988493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:03.988779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:03.989041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] PATH_INFO: `/v3/users/76a7ffd5f76f447fa517cfef267c842a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:03.990226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a7ffd5f76f447fa517cfef267c842a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:03.992253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:04.003056 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:04.041510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a7ffd5f76f447fa517cfef267c842a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.042306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a7ffd5f76f447fa517cfef267c842a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.050107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a7ffd5f76f447fa517cfef267c842a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.063366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a7ffd5f76f447fa517cfef267c842a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.064086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a7ffd5f76f447fa517cfef267c842a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.077938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1169/2317] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:30:03 2022] DELETE /identity/v3/users/76a7ffd5f76f447fa517cfef267c842a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:30:04.082520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153e7de-d283-474a-a5fa-c87ae17ab831 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:04.082980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153e7de-d283-474a-a5fa-c87ae17ab831 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:04.083127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153e7de-d283-474a-a5fa-c87ae17ab831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:04.103492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5153e7de-d283-474a-a5fa-c87ae17ab831 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:30:04.104739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5153e7de-d283-474a-a5fa-c87ae17ab831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:04.174900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1149/2318] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:30:04.184704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:04.186136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:04.254022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c514d-6649-4212-aacd-ea34083253ff admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:04.258384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:04.259036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:04.259640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:04.259776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:04.276015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:04.277666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:04.290582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1170/2319] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:30:04.573581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153e7de-d283-474a-a5fa-c87ae17ab831 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:04.575115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5153e7de-d283-474a-a5fa-c87ae17ab831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:04.589726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153e7de-d283-474a-a5fa-c87ae17ab831 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:04.594235 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:04.595776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:04.596427 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:04.596736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] PATH_INFO: `/v3/projects/664ab4816a9f4151b81de70e94b49a5e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:04.597952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=664ab4816a9f4151b81de70e94b49a5e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:04.599904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:04.646301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664ab4816a9f4151b81de70e94b49a5e', 'request_id': 'req-420c6e89-0e38-4555-ab66-376d76556f4e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.654113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664ab4816a9f4151b81de70e94b49a5e', 'request_id': 'req-420c6e89-0e38-4555-ab66-376d76556f4e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.661024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664ab4816a9f4151b81de70e94b49a5e', 'request_id': 'req-420c6e89-0e38-4555-ab66-376d76556f4e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.672053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] The token cache is being invalidate because project 664ab4816a9f4151b81de70e94b49a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:30:04.672342 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.673136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:04.674671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1150/2320] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:30:04 2022] DELETE /identity/v3/projects/664ab4816a9f4151b81de70e94b49a5e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:30:14.285286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:14.668980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:26.504407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:26.505797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:26.517796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394db67-505a-4d74-a870-7451db9e994c service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:26.566202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:26.566778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:26.567059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:26.567352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:26.569911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:26.652899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:26.654334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:26.674304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1171/2321] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:30:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:30:28.088978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:28.090486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:28.102655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c6e89-0e38-4555-ab66-376d76556f4e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:28.106158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:28.106790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:28.107069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:28.107403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:28.109077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:28.122474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:28.123933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:28.135645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1151/2322] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:30:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:30:29.904051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:29.905612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:29.974033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486249-cab0-4fbe-84df-36bb112b6783 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:29.978491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:29.979392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:29.979678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:29.979952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] PATH_INFO: `/v3/users/b6a6d8ab7cc04e14b8f148412974e995` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:29.981132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a6d8ab7cc04e14b8f148412974e995)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:29.983086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:30.010054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:30.038730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a6d8ab7cc04e14b8f148412974e995'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.039628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a6d8ab7cc04e14b8f148412974e995'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.048169 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a6d8ab7cc04e14b8f148412974e995'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.062449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a6d8ab7cc04e14b8f148412974e995'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.063273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-aaca593b-f108-42da-9b16-31dbb9996f6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a6d8ab7cc04e14b8f148412974e995'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.076922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1172/2323] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:30:29 2022] DELETE /identity/v3/users/b6a6d8ab7cc04e14b8f148412974e995 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:30:30.243996 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:30.255045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e888e8-0972-4dd2-9e03-15c236c3c033 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:30.315272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:30.316113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.316418 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.316698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] PATH_INFO: `/v3/projects/dd6efc6e9dad4ab1b9c40ab78d5a360d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.317899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6efc6e9dad4ab1b9c40ab78d5a360d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:30.319944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:30.370292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6efc6e9dad4ab1b9c40ab78d5a360d', 'request_id': 'req-e04b8be9-419c-4084-a586-05d444ea8b30'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.378591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6efc6e9dad4ab1b9c40ab78d5a360d', 'request_id': 'req-e04b8be9-419c-4084-a586-05d444ea8b30'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.386275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6efc6e9dad4ab1b9c40ab78d5a360d', 'request_id': 'req-e04b8be9-419c-4084-a586-05d444ea8b30'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.397376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] The token cache is being invalidate because project dd6efc6e9dad4ab1b9c40ab78d5a360d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:30:30.397716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.398450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e04b8be9-419c-4084-a586-05d444ea8b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:30:30.400120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1152/2324] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:30:30 2022] DELETE /identity/v3/projects/dd6efc6e9dad4ab1b9c40ab78d5a360d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:30:30.406120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac497a-b807-4629-9cc1-af554e16bde1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.406411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac497a-b807-4629-9cc1-af554e16bde1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.406685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac497a-b807-4629-9cc1-af554e16bde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.424260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f7ac497a-b807-4629-9cc1-af554e16bde1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:30:30.426659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f7ac497a-b807-4629-9cc1-af554e16bde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:30.528245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1173/2325] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:30:30.531061 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2510ad-a2bb-4cf7-bcd8-5700dd4b9e14 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.531399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2510ad-a2bb-4cf7-bcd8-5700dd4b9e14 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.531899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2510ad-a2bb-4cf7-bcd8-5700dd4b9e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.575754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2f2510ad-a2bb-4cf7-bcd8-5700dd4b9e14 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:30:30.577022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2f2510ad-a2bb-4cf7-bcd8-5700dd4b9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:30.596387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1153/2326] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:30:30.599061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac497a-b807-4629-9cc1-af554e16bde1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:30.613583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac497a-b807-4629-9cc1-af554e16bde1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:30.624081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:30.624849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.625215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.625562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.626662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:30:30.627034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:30.628960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:30.644529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1174/2327] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:30:30.646495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2510ad-a2bb-4cf7-bcd8-5700dd4b9e14 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:30.656042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2510ad-a2bb-4cf7-bcd8-5700dd4b9e14 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:30.660740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea33a48-add6-4180-b848-1c569f0681c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:30.661422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea33a48-add6-4180-b848-1c569f0681c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.661701 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea33a48-add6-4180-b848-1c569f0681c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.661977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea33a48-add6-4180-b848-1c569f0681c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.662710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea33a48-add6-4180-b848-1c569f0681c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:30.665278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea33a48-add6-4180-b848-1c569f0681c6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:30.705397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1154/2328] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:30:30 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:30:30.709517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:30.726571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a80d0d-3c95-4dfa-9ff9-fb590882bfe7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:30.795156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256e29-0b85-4ae6-8505-c2e4912394e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:30.795980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256e29-0b85-4ae6-8505-c2e4912394e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.796317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256e29-0b85-4ae6-8505-c2e4912394e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.796667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256e29-0b85-4ae6-8505-c2e4912394e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.797470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256e29-0b85-4ae6-8505-c2e4912394e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:30.799425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256e29-0b85-4ae6-8505-c2e4912394e2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:30.824785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1175/2329] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:30:30 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:30:30.828895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea33a48-add6-4180-b848-1c569f0681c6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:30.840431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea33a48-add6-4180-b848-1c569f0681c6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:30.845366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea55bbe-cb20-4139-9a31-cca278aa3704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:30.846115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea55bbe-cb20-4139-9a31-cca278aa3704 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.846466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea55bbe-cb20-4139-9a31-cca278aa3704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.846810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea55bbe-cb20-4139-9a31-cca278aa3704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.847655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea55bbe-cb20-4139-9a31-cca278aa3704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:30.849546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea55bbe-cb20-4139-9a31-cca278aa3704 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:30.864352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1155/2330] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:30:30.868785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256e29-0b85-4ae6-8505-c2e4912394e2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:30.879917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256e29-0b85-4ae6-8505-c2e4912394e2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:30.886084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab6d6d-bf1a-474e-93e3-f645b31fa35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:30.886760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6d6d-bf1a-474e-93e3-f645b31fa35b admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.887035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6d6d-bf1a-474e-93e3-f645b31fa35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.887306 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6d6d-bf1a-474e-93e3-f645b31fa35b admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.888078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6d6d-bf1a-474e-93e3-f645b31fa35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:30.889681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6d6d-bf1a-474e-93e3-f645b31fa35b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:30.901301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1176/2331] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:30:30.905271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea55bbe-cb20-4139-9a31-cca278aa3704 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:30.915925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea55bbe-cb20-4139-9a31-cca278aa3704 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:30.921689 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:30.922757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.923105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.923502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] PATH_INFO: `/v3/projects/c1187f363c0d40ffaebe938d3201e0ad/users/10c83f7ad6e94f34b7bd9a335ee52257/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.939606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1187f363c0d40ffaebe938d3201e0ad, user_id=10c83f7ad6e94f34b7bd9a335ee52257, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:30.942483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:30.957291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1156/2332] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:30:30 2022] PUT /identity/v3/projects/c1187f363c0d40ffaebe938d3201e0ad/users/10c83f7ad6e94f34b7bd9a335ee52257/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:30:30.963151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb018-9998-4daf-9340-e1379af26121 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:30.963766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb018-9998-4daf-9340-e1379af26121 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:30.963873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb018-9998-4daf-9340-e1379af26121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:30.978037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:30.984727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-956eb018-9998-4daf-9340-e1379af26121 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:30:30.986579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-956eb018-9998-4daf-9340-e1379af26121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:31.056391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1177/2333] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:30:31.071467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:31.073408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:31.094048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:31.145208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d14e48-072e-489e-8a5f-3b19bc094700 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:31.149794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df611bda-bd35-42f4-99b5-e1cc36e06102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:31.150568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df611bda-bd35-42f4-99b5-e1cc36e06102 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:31.150861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df611bda-bd35-42f4-99b5-e1cc36e06102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:31.151135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df611bda-bd35-42f4-99b5-e1cc36e06102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:31.160613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df611bda-bd35-42f4-99b5-e1cc36e06102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:31.162067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df611bda-bd35-42f4-99b5-e1cc36e06102 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:31.172974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1157/2334] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:30:35.923351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e17f4-f87d-48dc-b5b0-047040393243 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:35.923826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e17f4-f87d-48dc-b5b0-047040393243 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:35.924178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e17f4-f87d-48dc-b5b0-047040393243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:35.957220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d38e17f4-f87d-48dc-b5b0-047040393243 None None] MFA Rules not processed for user `10c83f7ad6e94f34b7bd9a335ee52257`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:30:35.958444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d38e17f4-f87d-48dc-b5b0-047040393243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:36.069803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1178/2335] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:30:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:30:36.075814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df611bda-bd35-42f4-99b5-e1cc36e06102 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:36.088867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df611bda-bd35-42f4-99b5-e1cc36e06102 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:36.095833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36753f-2a4d-43f6-9f94-e69fa86691c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:36.096601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36753f-2a4d-43f6-9f94-e69fa86691c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:36.098101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36753f-2a4d-43f6-9f94-e69fa86691c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:36.098655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36753f-2a4d-43f6-9f94-e69fa86691c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:36.099614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36753f-2a4d-43f6-9f94-e69fa86691c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:36.104105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36753f-2a4d-43f6-9f94-e69fa86691c5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:36.138347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1158/2336] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:30:36 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:30:36.141968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e17f4-f87d-48dc-b5b0-047040393243 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:36.154244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e17f4-f87d-48dc-b5b0-047040393243 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:36.223432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc137301-dbbf-4cfb-b88e-16977a77431b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:36.224101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc137301-dbbf-4cfb-b88e-16977a77431b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:36.224380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc137301-dbbf-4cfb-b88e-16977a77431b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:36.224652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc137301-dbbf-4cfb-b88e-16977a77431b admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:36.225392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc137301-dbbf-4cfb-b88e-16977a77431b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:36.228032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc137301-dbbf-4cfb-b88e-16977a77431b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:36.249834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1179/2337] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:30:36 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:30:36.253746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36753f-2a4d-43f6-9f94-e69fa86691c5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:36.264473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36753f-2a4d-43f6-9f94-e69fa86691c5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:36.269301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7903d1-99ce-42c7-8d0c-3bf34f7ec490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:36.269969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7903d1-99ce-42c7-8d0c-3bf34f7ec490 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:36.270254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7903d1-99ce-42c7-8d0c-3bf34f7ec490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:36.270579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7903d1-99ce-42c7-8d0c-3bf34f7ec490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:36.271315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7903d1-99ce-42c7-8d0c-3bf34f7ec490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:36.273614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7903d1-99ce-42c7-8d0c-3bf34f7ec490 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:36.284708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1159/2338] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:30:36.289495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc137301-dbbf-4cfb-b88e-16977a77431b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:36.300685 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc137301-dbbf-4cfb-b88e-16977a77431b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:36.305630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b2b9b-e838-42bf-a902-bb668ff974cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:36.306515 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b2b9b-e838-42bf-a902-bb668ff974cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:36.306789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b2b9b-e838-42bf-a902-bb668ff974cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:36.307060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b2b9b-e838-42bf-a902-bb668ff974cf admin admin] PATH_INFO: `/v3/projects/619f6038fc644e42bc968395310b90ba/users/73dc79b41df1408db9735928da6653df/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:36.322516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b2b9b-e838-42bf-a902-bb668ff974cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=619f6038fc644e42bc968395310b90ba, user_id=73dc79b41df1408db9735928da6653df, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:36.325208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b2b9b-e838-42bf-a902-bb668ff974cf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:36.340086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1180/2339] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:30:36 2022] PUT /identity/v3/projects/619f6038fc644e42bc968395310b90ba/users/73dc79b41df1408db9735928da6653df/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:30:36.344193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7903d1-99ce-42c7-8d0c-3bf34f7ec490 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:36.355282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7903d1-99ce-42c7-8d0c-3bf34f7ec490 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:36.422963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ea537-419e-4c6c-bafb-dffa15d89abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:36.423830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ea537-419e-4c6c-bafb-dffa15d89abc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:36.424052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ea537-419e-4c6c-bafb-dffa15d89abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:36.424391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ea537-419e-4c6c-bafb-dffa15d89abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:36.425056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ea537-419e-4c6c-bafb-dffa15d89abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:36.426881 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ea537-419e-4c6c-bafb-dffa15d89abc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:36.443079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1160/2340] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:30:36.445509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b2b9b-e838-42bf-a902-bb668ff974cf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:36.451717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:36.463714 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b2b9b-e838-42bf-a902-bb668ff974cf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:36.470164 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:36.471060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:36.471354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:36.471660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] PATH_INFO: `/v3/projects/619f6038fc644e42bc968395310b90ba/users/73dc79b41df1408db9735928da6653df/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:36.473621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] RBAC: Authorizing `identity:create_grant(project_id=619f6038fc644e42bc968395310b90ba, user_id=73dc79b41df1408db9735928da6653df, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:36.477092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:36.490192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1181/2341] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:30:36 2022] PUT /identity/v3/projects/619f6038fc644e42bc968395310b90ba/users/73dc79b41df1408db9735928da6653df/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:30:42.120912 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722833f-adab-495b-b198-0c0377d08b6e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:42.121441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722833f-adab-495b-b198-0c0377d08b6e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:42.121541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722833f-adab-495b-b198-0c0377d08b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:42.156874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6722833f-adab-495b-b198-0c0377d08b6e None None] MFA Rules not processed for user `73dc79b41df1408db9735928da6653df`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:30:42.163736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6722833f-adab-495b-b198-0c0377d08b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:42.264017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1161/2342] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:30:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:30:42.280347 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:42.284157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:42.392334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a1c99-89f6-42a3-a1e2-be6edf08cbec admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:42.407067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ec086-5348-4d80-ab2a-a29ce790ef3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:42.407803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ec086-5348-4d80-ab2a-a29ce790ef3b service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:42.408070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ec086-5348-4d80-ab2a-a29ce790ef3b service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:42.408350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ec086-5348-4d80-ab2a-a29ce790ef3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:42.532022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ec086-5348-4d80-ab2a-a29ce790ef3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:42.533898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ec086-5348-4d80-ab2a-a29ce790ef3b service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:42.547024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1182/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:30:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:30:43.557643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455891-063e-42e4-af91-16fb5cc03197 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:43.558281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455891-063e-42e4-af91-16fb5cc03197 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:43.558394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455891-063e-42e4-af91-16fb5cc03197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:43.565631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:43.577671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-87455891-063e-42e4-af91-16fb5cc03197 None None] MFA Rules not processed for user `10c83f7ad6e94f34b7bd9a335ee52257`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:30:43.578876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-87455891-063e-42e4-af91-16fb5cc03197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:43.587588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1162/2344] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:30:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:30:44.009238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a335b-76b7-43ec-b203-27dd881e0fdd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:44.009794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a335b-76b7-43ec-b203-27dd881e0fdd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:44.010213 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a335b-76b7-43ec-b203-27dd881e0fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:44.030084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-464a335b-76b7-43ec-b203-27dd881e0fdd None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:30:44.031573 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-464a335b-76b7-43ec-b203-27dd881e0fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:44.099389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1183/2345] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:30:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:30:44.137784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455891-063e-42e4-af91-16fb5cc03197 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:44.140919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-87455891-063e-42e4-af91-16fb5cc03197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:44.238682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455891-063e-42e4-af91-16fb5cc03197 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:44.244040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c3fd6-a903-4a53-ac1d-302a7e228bbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:44.244824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c3fd6-a903-4a53-ac1d-302a7e228bbf service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:44.245312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c3fd6-a903-4a53-ac1d-302a7e228bbf service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:44.245756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c3fd6-a903-4a53-ac1d-302a7e228bbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:44.256023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c3fd6-a903-4a53-ac1d-302a7e228bbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:44.258609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c3fd6-a903-4a53-ac1d-302a7e228bbf service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:44.268148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1163/2346] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:30:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 17:30:54.092109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:54.265124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=33 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:30:56.762960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a335b-76b7-43ec-b203-27dd881e0fdd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:30:56.764511 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-464a335b-76b7-43ec-b203-27dd881e0fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:56.819749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a335b-76b7-43ec-b203-27dd881e0fdd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:30:56.824059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3480b1f-52a8-47c9-90fa-a7efa150e6e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:30:56.824700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480b1f-52a8-47c9-90fa-a7efa150e6e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:30:56.824979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480b1f-52a8-47c9-90fa-a7efa150e6e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:30:56.825254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480b1f-52a8-47c9-90fa-a7efa150e6e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:30:56.827110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f3480b1f-52a8-47c9-90fa-a7efa150e6e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:30:56.837145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3480b1f-52a8-47c9-90fa-a7efa150e6e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:30:56.838664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3480b1f-52a8-47c9-90fa-a7efa150e6e9 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:30:56.847165 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1184/2347] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:30:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:31:01.352097 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c3fd6-a903-4a53-ac1d-302a7e228bbf service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:01.362531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c3fd6-a903-4a53-ac1d-302a7e228bbf service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:01.366410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8c063-eea1-4b2f-9971-5c2a44199269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:01.367082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c063-eea1-4b2f-9971-5c2a44199269 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.367394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c063-eea1-4b2f-9971-5c2a44199269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.367714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c063-eea1-4b2f-9971-5c2a44199269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.369754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-08d8c063-eea1-4b2f-9971-5c2a44199269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:01.380800 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8c063-eea1-4b2f-9971-5c2a44199269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:01.382412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8c063-eea1-4b2f-9971-5c2a44199269 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:01.392617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1164/2348] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:01.511515 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae65cb-0e97-4516-b958-2987bfe8538f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.512068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae65cb-0e97-4516-b958-2987bfe8538f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.512153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae65cb-0e97-4516-b958-2987bfe8538f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.513987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1185/2349] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:31:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:31:01.520497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec8e8-ae71-4e7f-895c-6c1ee07028c2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.520971 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec8e8-ae71-4e7f-895c-6c1ee07028c2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.521264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec8e8-ae71-4e7f-895c-6c1ee07028c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.571703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-56cec8e8-ae71-4e7f-895c-6c1ee07028c2 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:31:01.572940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-56cec8e8-ae71-4e7f-895c-6c1ee07028c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:01.585040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1165/2350] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:31:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:31:01.603244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae65cb-0e97-4516-b958-2987bfe8538f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:01.605446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a5ae65cb-0e97-4516-b958-2987bfe8538f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:01.711068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae65cb-0e97-4516-b958-2987bfe8538f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:01.715104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f2062-57e4-41f5-85ef-c0258e5820ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:01.715914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2062-57e4-41f5-85ef-c0258e5820ca service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.716138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2062-57e4-41f5-85ef-c0258e5820ca service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.716415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2062-57e4-41f5-85ef-c0258e5820ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.727852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f2062-57e4-41f5-85ef-c0258e5820ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:01.729304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f2062-57e4-41f5-85ef-c0258e5820ca service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:01.740835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1186/2351] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:01.809716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5cfb0-99cc-41a4-a5b3-0f9479260ef2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.810020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5cfb0-99cc-41a4-a5b3-0f9479260ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.810259 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5cfb0-99cc-41a4-a5b3-0f9479260ef2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.811666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1166/2352] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:31:01.816701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36159b8-5164-427f-ab86-e8f0b4019f96 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.816979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36159b8-5164-427f-ab86-e8f0b4019f96 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.817253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36159b8-5164-427f-ab86-e8f0b4019f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.836283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f36159b8-5164-427f-ab86-e8f0b4019f96 None None] MFA Rules not processed for user `71a48b3b4fa94798b158488f08bae743`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:31:01.837486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f36159b8-5164-427f-ab86-e8f0b4019f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:01.843968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1187/2353] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:31:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:31:01.855681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5cfb0-99cc-41a4-a5b3-0f9479260ef2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:01.866437 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5cfb0-99cc-41a4-a5b3-0f9479260ef2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:01.870522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cf1cc-994a-49ff-a2c3-4fbd6135396d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:01.871153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf1cc-994a-49ff-a2c3-4fbd6135396d service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.871475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf1cc-994a-49ff-a2c3-4fbd6135396d service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.871763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf1cc-994a-49ff-a2c3-4fbd6135396d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.879018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cf1cc-994a-49ff-a2c3-4fbd6135396d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:01.880463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cf1cc-994a-49ff-a2c3-4fbd6135396d service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:01.888851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1167/2354] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:01.971723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0130-80c3-41d7-9f77-776898fe3a04 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.971932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0130-80c3-41d7-9f77-776898fe3a04 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.972208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0130-80c3-41d7-9f77-776898fe3a04 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.973571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1188/2355] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:31:01.979382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf582f-ec5d-4f70-8d10-ca7e1e33a342 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:01.979676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf582f-ec5d-4f70-8d10-ca7e1e33a342 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:01.979955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf582f-ec5d-4f70-8d10-ca7e1e33a342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:01.998555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bbcf582f-ec5d-4f70-8d10-ca7e1e33a342 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:31:01.999847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bbcf582f-ec5d-4f70-8d10-ca7e1e33a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:02.010348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1168/2356] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:31:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:31:02.020567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba0130-80c3-41d7-9f77-776898fe3a04 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:02.031551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba0130-80c3-41d7-9f77-776898fe3a04 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:02.035022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d0526-c67c-4f0f-b8fd-1956b9db720d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:02.035706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0526-c67c-4f0f-b8fd-1956b9db720d service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:02.035983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0526-c67c-4f0f-b8fd-1956b9db720d service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:02.036253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0526-c67c-4f0f-b8fd-1956b9db720d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:02.046682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d0526-c67c-4f0f-b8fd-1956b9db720d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:02.048142 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d0526-c67c-4f0f-b8fd-1956b9db720d service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:02.059640 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1189/2357] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:02.121733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf582f-ec5d-4f70-8d10-ca7e1e33a342 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:02.123255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bbcf582f-ec5d-4f70-8d10-ca7e1e33a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:02.191301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf582f-ec5d-4f70-8d10-ca7e1e33a342 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:02.196510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:02.197155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:02.197439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:02.197714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:02.199510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:02.210551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:02.212280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:02.222299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1169/2358] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:05.136860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4597-47ca-4b9b-a260-6f1cfa8af0f9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:05.137286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4597-47ca-4b9b-a260-6f1cfa8af0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:05.137412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4597-47ca-4b9b-a260-6f1cfa8af0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:05.155964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bcdb4597-47ca-4b9b-a260-6f1cfa8af0f9 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:31:05.157178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bcdb4597-47ca-4b9b-a260-6f1cfa8af0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:05.171616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1190/2359] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:31:05.178090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:05.191766 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d8419-ea1c-4ee0-9ca9-79f4a11ea2f6 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:05.197068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:05.197722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:05.198002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:05.198274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:05.207922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:05.209510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:05.218821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1170/2360] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:31:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 17:31:08.876666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb4597-47ca-4b9b-a260-6f1cfa8af0f9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:08.887312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb4597-47ca-4b9b-a260-6f1cfa8af0f9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:08.891099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:08.892189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:08.892546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:08.892893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:08.894792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:08.907033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:08.908633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:08.918206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1191/2361] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:15.215564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=62 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:31:18.913737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=79 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:31:20.003910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:20.005255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:20.020946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a229db-a042-4cf0-a8e5-f3be59bd51ac service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:20.026370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:20.027004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:20.027290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:20.027595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:20.029320 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:20.086592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:20.088204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:20.097076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1171/2362] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:23.672420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:23.692432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867e189-7c4b-401b-807a-717520cbd5cb service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:23.700830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:23.701562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:23.702213 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:23.702491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:23.704557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:23.724942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:23.727713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:23.743204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1192/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:30.093010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=28 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:31:31.273877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:31.285032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8640f-b383-45da-bbe3-fb7235170ccb service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:31.291420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34066d08-716f-435d-a9d9-fc8f75585876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:31.292088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34066d08-716f-435d-a9d9-fc8f75585876 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:31.292938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34066d08-716f-435d-a9d9-fc8f75585876 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:31.293223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34066d08-716f-435d-a9d9-fc8f75585876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:31.295406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-34066d08-716f-435d-a9d9-fc8f75585876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:31.353651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:31.364185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34066d08-716f-435d-a9d9-fc8f75585876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:31.366370 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34066d08-716f-435d-a9d9-fc8f75585876 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:31.369245 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a3eda-2790-4701-91f8-399fca70f382 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:31.375624 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:31.376434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:31.376819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:31.377199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:31.378300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1172/2364] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:31.379569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:31.398718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:31.400492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:31.415787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1193/2365] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:34.611432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a9b64-f3b7-4a1f-b1f6-6ff76f72d6fc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:34.611954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a9b64-f3b7-4a1f-b1f6-6ff76f72d6fc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:34.611954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a9b64-f3b7-4a1f-b1f6-6ff76f72d6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:34.631014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a95a9b64-f3b7-4a1f-b1f6-6ff76f72d6fc None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:31:34.632990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a95a9b64-f3b7-4a1f-b1f6-6ff76f72d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:34.640458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1173/2366] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 17:31:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:31:34.652677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:34.665304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015910e8-6397-41a6-adf9-b363098aa24a service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:34.670352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f197a5b-ae68-49e5-8a99-e56b9001b788 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:34.671020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f197a5b-ae68-49e5-8a99-e56b9001b788 service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:34.671302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f197a5b-ae68-49e5-8a99-e56b9001b788 service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:34.671612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f197a5b-ae68-49e5-8a99-e56b9001b788 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:34.680869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f197a5b-ae68-49e5-8a99-e56b9001b788 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:34.682470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f197a5b-ae68-49e5-8a99-e56b9001b788 service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:34.690825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1194/2367] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:31:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:31:44.086897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a9b64-f3b7-4a1f-b1f6-6ff76f72d6fc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:44.096127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a9b64-f3b7-4a1f-b1f6-6ff76f72d6fc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:44.099667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2868e-05b4-4850-9cd9-dc8dcc8bfedf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:44.100305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2868e-05b4-4850-9cd9-dc8dcc8bfedf service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:44.100588 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2868e-05b4-4850-9cd9-dc8dcc8bfedf service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:44.100868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2868e-05b4-4850-9cd9-dc8dcc8bfedf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:44.110650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2868e-05b4-4850-9cd9-dc8dcc8bfedf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:44.112142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2868e-05b4-4850-9cd9-dc8dcc8bfedf service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:44.121736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1174/2368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:44.687601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=18 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:31:46.039094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f197a5b-ae68-49e5-8a99-e56b9001b788 service swift] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:31:46.052145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f197a5b-ae68-49e5-8a99-e56b9001b788 service swift] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:31:46.058597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:31:46.059643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:46.060009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:46.060357 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:46.062269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:46.076952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:31:46.078664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:31:46.091307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1195/2369] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:31:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:31:54.117282 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=37 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:31:54.396609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1cb8-7063-476d-9ad7-1dfdda3c28f5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:31:54.396906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1cb8-7063-476d-9ad7-1dfdda3c28f5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:31:54.397180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1cb8-7063-476d-9ad7-1dfdda3c28f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:31:54.416844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3fde1cb8-7063-476d-9ad7-1dfdda3c28f5 None None] MFA Rules not processed for user `927f21ef1ca74ec58f85181728933e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:31:54.418064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3fde1cb8-7063-476d-9ad7-1dfdda3c28f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:31:54.425803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1175/2370] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:31:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:31:56.085608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=6 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:32:04.410328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:32:37.881528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:37.889564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde1cb8-7063-476d-9ad7-1dfdda3c28f5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:37.890085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863ad55-b991-472f-a8e6-9cd544e9889d service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:37.893588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255eb7a-c417-468c-9c98-aefd52782b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:37.894226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255eb7a-c417-468c-9c98-aefd52782b6f service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:37.894501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255eb7a-c417-468c-9c98-aefd52782b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:37.894897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255eb7a-c417-468c-9c98-aefd52782b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:37.896927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a255eb7a-c417-468c-9c98-aefd52782b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:37.902669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde1cb8-7063-476d-9ad7-1dfdda3c28f5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:37.908196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:37.908839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:37.909119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:37.909392 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:37.911129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:37.958723 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255eb7a-c417-468c-9c98-aefd52782b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:37.963715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255eb7a-c417-468c-9c98-aefd52782b6f service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:37.970866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:37.973263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:37.978262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1196/2371] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:32:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:32:37.984456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1176/2372] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:32:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:32:40.358105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b012-7fd3-44ec-9510-16529e564b46 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:40.358579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b012-7fd3-44ec-9510-16529e564b46 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:40.358684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b012-7fd3-44ec-9510-16529e564b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:40.376429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8f12b012-7fd3-44ec-9510-16529e564b46 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:32:40.377639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8f12b012-7fd3-44ec-9510-16529e564b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:40.385160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1197/2373] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 17:32:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:32:40.446822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:40.459462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334abce4-8f15-46d9-9e55-c85fd7f9c8fe service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:40.464622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f14753-a592-41e8-a204-1a8458ed9052 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:40.465354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14753-a592-41e8-a204-1a8458ed9052 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:40.465708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14753-a592-41e8-a204-1a8458ed9052 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:40.466075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14753-a592-41e8-a204-1a8458ed9052 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:40.474805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f14753-a592-41e8-a204-1a8458ed9052 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:40.476430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f14753-a592-41e8-a204-1a8458ed9052 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:40.485106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1177/2374] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:32:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:32:40.828582 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12b012-7fd3-44ec-9510-16529e564b46 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:40.837954 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12b012-7fd3-44ec-9510-16529e564b46 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:40.841481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:40.842160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:40.842523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:40.842803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:40.844725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:40.911508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:40.912952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:40.923419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1198/2375] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:32:43.119134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f14753-a592-41e8-a204-1a8458ed9052 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:43.133468 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f14753-a592-41e8-a204-1a8458ed9052 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:43.139391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:43.140142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:43.140496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:43.140854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:43.142737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:43.218558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:43.220198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:43.243561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1178/2376] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:32:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:32:44.185221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:44.187484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:44.202055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08f652-e574-4654-8d77-537d5176b0f3 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:44.207315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:44.208161 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:44.208453 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:44.208730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:44.210680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:44.226357 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:44.228369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:44.243149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1199/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:32:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:32:47.990715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:47.992330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:48.009659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eb222-54f2-4dd8-a3ee-05862977ca58 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:48.015745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:48.016662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:48.016940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:48.017210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] PATH_INFO: `/v3/users/681c8dec79764371badd23608ff4fa5a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:48.018428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] RBAC: Authorizing `identity:delete_user(user_id=681c8dec79764371badd23608ff4fa5a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:48.020415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:48.079520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '681c8dec79764371badd23608ff4fa5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.094023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '681c8dec79764371badd23608ff4fa5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.115700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '681c8dec79764371badd23608ff4fa5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.116850 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '681c8dec79764371badd23608ff4fa5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.130782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:32:48.133803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '681c8dec79764371badd23608ff4fa5a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.143878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1179/2378] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:32:47 2022] DELETE /identity/v3/users/681c8dec79764371badd23608ff4fa5a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:32:48.340399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:48.355072 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff75214-b158-40db-8e52-2c8270f04bf3 service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:48.422643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:48.423653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:48.423931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:48.424202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] PATH_INFO: `/v3/projects/33d09d2e02c945a09e79efae8d8ab96c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:48.425416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d09d2e02c945a09e79efae8d8ab96c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:48.427384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:48.444096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:32:48.475445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d09d2e02c945a09e79efae8d8ab96c', 'request_id': 'req-190c347c-5700-487d-a522-777f98ca5ac7'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.481562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d09d2e02c945a09e79efae8d8ab96c', 'request_id': 'req-190c347c-5700-487d-a522-777f98ca5ac7'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.489557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d09d2e02c945a09e79efae8d8ab96c', 'request_id': 'req-190c347c-5700-487d-a522-777f98ca5ac7'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.502975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] The token cache is being invalidate because project 33d09d2e02c945a09e79efae8d8ab96c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:32:48.503262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.504264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.506497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1200/2379] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:32:48 2022] DELETE /identity/v3/projects/33d09d2e02c945a09e79efae8d8ab96c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:32:48.510538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:48.511713 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:48.526995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85318b-37bf-4e85-9e78-5604b263873d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:48.592882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:48.593946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:48.594304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:48.594653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] PATH_INFO: `/v3/users/11639489d3c746068a47376743df065b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:48.596043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11639489d3c746068a47376743df065b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:48.598356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:48.658675 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11639489d3c746068a47376743df065b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.666954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11639489d3c746068a47376743df065b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.681223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11639489d3c746068a47376743df065b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.681943 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11639489d3c746068a47376743df065b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.690730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-76c95668-d9d9-44da-b690-9bff6916d1f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11639489d3c746068a47376743df065b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:48.697605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1180/2380] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:32:48 2022] DELETE /identity/v3/users/11639489d3c746068a47376743df065b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:32:48.902149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:32:48.905855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:48.934608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c347c-5700-487d-a522-777f98ca5ac7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:32:49.051931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:32:49.053143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:49.053565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:49.055373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] PATH_INFO: `/v3/projects/9e10e1fb9f6f425cb76033446d29aa8b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:49.055373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e10e1fb9f6f425cb76033446d29aa8b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:32:49.057220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:32:49.117247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e10e1fb9f6f425cb76033446d29aa8b', 'request_id': 'req-fca08c33-1b52-44f9-9e71-47c5b59e0bac'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:49.123299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e10e1fb9f6f425cb76033446d29aa8b', 'request_id': 'req-fca08c33-1b52-44f9-9e71-47c5b59e0bac'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:49.130845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e10e1fb9f6f425cb76033446d29aa8b', 'request_id': 'req-fca08c33-1b52-44f9-9e71-47c5b59e0bac'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:49.150461 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] The token cache is being invalidate because project 9e10e1fb9f6f425cb76033446d29aa8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:32:49.150841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:49.151592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fca08c33-1b52-44f9-9e71-47c5b59e0bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:32:49.153380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1201/2381] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:32:48 2022] DELETE /identity/v3/projects/9e10e1fb9f6f425cb76033446d29aa8b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:32:50.798296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5231c1-57f7-4ce4-a636-94f0ef781015 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:32:50.798859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5231c1-57f7-4ce4-a636-94f0ef781015 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:32:50.799311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5231c1-57f7-4ce4-a636-94f0ef781015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:32:50.818230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ea5231c1-57f7-4ce4-a636-94f0ef781015 None None] MFA Rules not processed for user `710b406ba65646ed94c3f9d356b161f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:32:50.819458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ea5231c1-57f7-4ce4-a636-94f0ef781015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:32:50.874577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1181/2382] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:32:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:32:59.147385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=69,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:33:00.867107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:33:09.327949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd78645-3990-40e9-be39-4413cf19bd4e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:09.328458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd78645-3990-40e9-be39-4413cf19bd4e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:09.328568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd78645-3990-40e9-be39-4413cf19bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:09.346201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6cd78645-3990-40e9-be39-4413cf19bd4e None None] MFA Rules not processed for user `7fcc62869ac34381a2327be46511137b`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:33:09.347425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6cd78645-3990-40e9-be39-4413cf19bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:09.400980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1202/2383] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:33:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:33:19.394528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=25 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:33:24.949666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5231c1-57f7-4ce4-a636-94f0ef781015 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:24.951606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ea5231c1-57f7-4ce4-a636-94f0ef781015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:25.008793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5231c1-57f7-4ce4-a636-94f0ef781015 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:25.012517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:25.013169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:25.013455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:25.013729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:25.015551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:25.079972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:25.081921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:25.092075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1182/2384] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:33:29.048963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd78645-3990-40e9-be39-4413cf19bd4e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:29.052168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6cd78645-3990-40e9-be39-4413cf19bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:29.078854 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd78645-3990-40e9-be39-4413cf19bd4e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:29.083833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:29.084724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.085061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.085259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] PATH_INFO: `/v3/users/927f21ef1ca74ec58f85181728933e82` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.087030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=927f21ef1ca74ec58f85181728933e82)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:29.089166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:29.144358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927f21ef1ca74ec58f85181728933e82'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.145000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927f21ef1ca74ec58f85181728933e82'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.151889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927f21ef1ca74ec58f85181728933e82'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.164124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927f21ef1ca74ec58f85181728933e82'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.164753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f70cf4a-72c1-4f7f-b647-1d3621d04aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927f21ef1ca74ec58f85181728933e82'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.177800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1203/2385] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:33:29 2022] DELETE /identity/v3/users/927f21ef1ca74ec58f85181728933e82 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:33:29.347160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:29.357114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a5119-62a1-4e16-8f21-caf247139a0a service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:29.414435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:29.415300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.415611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.415885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] PATH_INFO: `/v3/projects/8e92a5dc387f47c99ce98da4d6fd9d92` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.417079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e92a5dc387f47c99ce98da4d6fd9d92)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:29.418976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:29.463829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e92a5dc387f47c99ce98da4d6fd9d92', 'request_id': 'req-172c60c4-f26e-4858-8e62-f1009b6e0478'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.466258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:33:29.472036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e92a5dc387f47c99ce98da4d6fd9d92', 'request_id': 'req-172c60c4-f26e-4858-8e62-f1009b6e0478'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.479744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e92a5dc387f47c99ce98da4d6fd9d92', 'request_id': 'req-172c60c4-f26e-4858-8e62-f1009b6e0478'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.490045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] The token cache is being invalidate because project 8e92a5dc387f47c99ce98da4d6fd9d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:33:29.490328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.490952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-172c60c4-f26e-4858-8e62-f1009b6e0478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:33:29.492501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1183/2386] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:33:29 2022] DELETE /identity/v3/projects/8e92a5dc387f47c99ce98da4d6fd9d92 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:33:29.497373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca17c3-e7c9-4dd6-b504-75a78b01da57 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.497660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca17c3-e7c9-4dd6-b504-75a78b01da57 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.497935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca17c3-e7c9-4dd6-b504-75a78b01da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.515200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6aca17c3-e7c9-4dd6-b504-75a78b01da57 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:33:29.516456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6aca17c3-e7c9-4dd6-b504-75a78b01da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:29.575313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1204/2387] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:33:29.583246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f2e7d-bfa9-48e0-9780-a4a7b0053492 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.583590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f2e7d-bfa9-48e0-9780-a4a7b0053492 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.583875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f2e7d-bfa9-48e0-9780-a4a7b0053492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.602474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0c0f2e7d-bfa9-48e0-9780-a4a7b0053492 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:33:29.603733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0c0f2e7d-bfa9-48e0-9780-a4a7b0053492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:29.611758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1184/2388] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:33:29.615646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca17c3-e7c9-4dd6-b504-75a78b01da57 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:29.625420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca17c3-e7c9-4dd6-b504-75a78b01da57 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:29.630322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:29.631276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.631588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.631863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.632590 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:33:29.632900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:29.634499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:29.647109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1205/2389] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:33:29.650868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f2e7d-bfa9-48e0-9780-a4a7b0053492 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:29.660551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f2e7d-bfa9-48e0-9780-a4a7b0053492 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:29.665119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77798d4d-6393-45d4-b372-b4f484480006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:29.665756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798d4d-6393-45d4-b372-b4f484480006 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.666035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798d4d-6393-45d4-b372-b4f484480006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.666318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798d4d-6393-45d4-b372-b4f484480006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.666994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77798d4d-6393-45d4-b372-b4f484480006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:29.668908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77798d4d-6393-45d4-b372-b4f484480006 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:29.705733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1185/2390] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:33:29 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:33:29.709981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:29.720225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214f7d7-0d14-4f88-bc7f-c246e001f8e2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:29.778301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc43e4-64a5-4d07-bf3b-37b28bf5a63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:29.778958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc43e4-64a5-4d07-bf3b-37b28bf5a63d admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.779235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc43e4-64a5-4d07-bf3b-37b28bf5a63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.779538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc43e4-64a5-4d07-bf3b-37b28bf5a63d admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.780211 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc43e4-64a5-4d07-bf3b-37b28bf5a63d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:29.782108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc43e4-64a5-4d07-bf3b-37b28bf5a63d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:29.789196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:33:29.806911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1206/2391] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:33:29 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:33:29.808248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77798d4d-6393-45d4-b372-b4f484480006 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:29.819105 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77798d4d-6393-45d4-b372-b4f484480006 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:29.823753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f438304-c1cd-4dd9-8731-229c7cb5d2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:29.824417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f438304-c1cd-4dd9-8731-229c7cb5d2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.824702 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f438304-c1cd-4dd9-8731-229c7cb5d2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.824981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f438304-c1cd-4dd9-8731-229c7cb5d2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.825812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f438304-c1cd-4dd9-8731-229c7cb5d2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:29.827359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f438304-c1cd-4dd9-8731-229c7cb5d2d8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:29.838069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1186/2392] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:33:29.842196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc43e4-64a5-4d07-bf3b-37b28bf5a63d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:29.852139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc43e4-64a5-4d07-bf3b-37b28bf5a63d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:29.856712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e6f23-4083-4e66-a311-88d13a09191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:29.857364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6f23-4083-4e66-a311-88d13a09191e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.857652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6f23-4083-4e66-a311-88d13a09191e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.857922 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6f23-4083-4e66-a311-88d13a09191e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.858664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e6f23-4083-4e66-a311-88d13a09191e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:29.860297 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e6f23-4083-4e66-a311-88d13a09191e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:29.870055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1207/2393] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:33:29.874079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f438304-c1cd-4dd9-8731-229c7cb5d2d8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:29.884839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f438304-c1cd-4dd9-8731-229c7cb5d2d8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:29.889472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:29.890348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.890631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.890908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] PATH_INFO: `/v3/projects/9525b35792a04dc3a6f109cb39ee9d50/users/7343386004e24474bc0af9126f47182b/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.908848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9525b35792a04dc3a6f109cb39ee9d50, user_id=7343386004e24474bc0af9126f47182b, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:29.911544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:29.924477 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1187/2394] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:33:29 2022] PUT /identity/v3/projects/9525b35792a04dc3a6f109cb39ee9d50/users/7343386004e24474bc0af9126f47182b/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:33:29.929677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b6e7d-5872-4dc6-a0bc-615dca214fdf None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:29.929973 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b6e7d-5872-4dc6-a0bc-615dca214fdf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:29.930256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b6e7d-5872-4dc6-a0bc-615dca214fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:29.948706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ec4b6e7d-5872-4dc6-a0bc-615dca214fdf None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:33:29.949906 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ec4b6e7d-5872-4dc6-a0bc-615dca214fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:30.008019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1208/2395] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:33:30.017469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:30.019875 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:30.076033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e149db-903e-44f0-afc0-13479fce3665 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:30.079644 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:30.080944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:30.081238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:30.081521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:30.090524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:30.091998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:30.101374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1188/2396] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:33:33.917307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7056df6-a462-44fe-95ea-d8e02b5df726 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:33.917841 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7056df6-a462-44fe-95ea-d8e02b5df726 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:33.917991 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7056df6-a462-44fe-95ea-d8e02b5df726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:33.946926 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e7056df6-a462-44fe-95ea-d8e02b5df726 None None] MFA Rules not processed for user `7343386004e24474bc0af9126f47182b`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:33:33.951851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e7056df6-a462-44fe-95ea-d8e02b5df726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:34.028618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1209/2397] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:33:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:33:34.043843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:33:34.045302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:34.122404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025baa8-aa3e-4185-874a-a6995efa6527 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:33:34.128308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:33:34.128972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:34.129261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:34.129538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:34.136058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:33:34.138846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:33:34.141094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:33:34.153864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1189/2398] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:33:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:33:35.514325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be942ade-9f07-457a-ac79-571b030eaa3c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:33:35.514810 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be942ade-9f07-457a-ac79-571b030eaa3c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:33:35.514908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be942ade-9f07-457a-ac79-571b030eaa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:33:35.536399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-be942ade-9f07-457a-ac79-571b030eaa3c None None] MFA Rules not processed for user `7343386004e24474bc0af9126f47182b`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:33:35.537636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-be942ade-9f07-457a-ac79-571b030eaa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:33:35.544718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1210/2399] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:33:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:33:44.150390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:33:45.529282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:34:25.495618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:34:25.497042 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:34:25.568032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccf0c8-4944-4ff3-9dc7-45497e0a30c2 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:34:25.573250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1df839-e63b-4a80-8c88-4564bc4de25d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:34:25.576558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1df839-e63b-4a80-8c88-4564bc4de25d service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:25.576847 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1df839-e63b-4a80-8c88-4564bc4de25d service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:25.577125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1df839-e63b-4a80-8c88-4564bc4de25d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:25.578951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e1df839-e63b-4a80-8c88-4564bc4de25d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:34:25.590793 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1df839-e63b-4a80-8c88-4564bc4de25d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:34:25.592470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1df839-e63b-4a80-8c88-4564bc4de25d service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:34:25.602415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1190/2400] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:34:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:34:26.979609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28640d4a-0f15-43b4-b4ef-354f834828ca None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:26.980181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28640d4a-0f15-43b4-b4ef-354f834828ca None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:26.980318 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28640d4a-0f15-43b4-b4ef-354f834828ca None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:26.981860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1211/2401] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:34:26.988769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a880-f810-40a4-b3cc-28c6473cc1fc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:26.989272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a880-f810-40a4-b3cc-28c6473cc1fc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:26.989380 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a880-f810-40a4-b3cc-28c6473cc1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:27.008009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e248a880-f810-40a4-b3cc-28c6473cc1fc None None] MFA Rules not processed for user `71a48b3b4fa94798b158488f08bae743`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:34:27.009223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e248a880-f810-40a4-b3cc-28c6473cc1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:34:27.015748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1191/2402] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:34:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:34:27.027651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28640d4a-0f15-43b4-b4ef-354f834828ca None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:34:27.038923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28640d4a-0f15-43b4-b4ef-354f834828ca None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:34:27.042427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed863cb0-9242-41dd-8c85-4becaeeae987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:34:27.043090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed863cb0-9242-41dd-8c85-4becaeeae987 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:27.043459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed863cb0-9242-41dd-8c85-4becaeeae987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:27.043681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed863cb0-9242-41dd-8c85-4becaeeae987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:27.051563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed863cb0-9242-41dd-8c85-4becaeeae987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:34:27.052990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed863cb0-9242-41dd-8c85-4becaeeae987 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:34:27.061200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1212/2403] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:34:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:34:27.144083 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25a7f-f814-44ca-8612-52200d99c29c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:27.144456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25a7f-f814-44ca-8612-52200d99c29c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:27.144742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25a7f-f814-44ca-8612-52200d99c29c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:27.146125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1192/2404] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 17:34:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 17:34:27.151474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f136ae0-91a9-4de9-8d9f-3209e7315da6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:27.151766 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f136ae0-91a9-4de9-8d9f-3209e7315da6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:27.152042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f136ae0-91a9-4de9-8d9f-3209e7315da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:27.170462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3f136ae0-91a9-4de9-8d9f-3209e7315da6 None None] MFA Rules not processed for user `5b9888d2731f4117b5fc2c3a720931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:34:27.171696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3f136ae0-91a9-4de9-8d9f-3209e7315da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:34:27.180920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1213/2405] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 17:34:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:34:27.191620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25a7f-f814-44ca-8612-52200d99c29c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:34:27.193357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-89b25a7f-f814-44ca-8612-52200d99c29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:34:27.206953 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25a7f-f814-44ca-8612-52200d99c29c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:34:27.210516 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:34:27.211152 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:27.211482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:27.211763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:27.222559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:34:27.224016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:34:27.234693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1193/2406] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:34:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:34:37.165254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=9 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:34:37.230119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=41 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:34:44.708160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348c27-98b1-491d-95d0-4df7773bb11f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:44.708659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348c27-98b1-491d-95d0-4df7773bb11f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:44.708755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348c27-98b1-491d-95d0-4df7773bb11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:44.732140 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-16348c27-98b1-491d-95d0-4df7773bb11f None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:34:44.733873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-16348c27-98b1-491d-95d0-4df7773bb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:34:44.792470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1214/2407] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:34:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:34:44.838674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:34:44.840115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:34:44.912828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e7c56-ffd0-4a7a-9680-1e42cb6a5ae9 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:34:44.917785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f0c22-d339-4118-a9fa-45ed191fe376 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:34:44.918439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0c22-d339-4118-a9fa-45ed191fe376 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:34:44.918722 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0c22-d339-4118-a9fa-45ed191fe376 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:34:44.919002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0c22-d339-4118-a9fa-45ed191fe376 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:34:44.927925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f0c22-d339-4118-a9fa-45ed191fe376 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:34:44.929511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f0c22-d339-4118-a9fa-45ed191fe376 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:34:44.937450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1194/2408] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:34:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:34:54.784614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=25 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:34:54.934219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=32 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:02.658783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c98db6-dd3a-47a9-a26f-32e045c522fc None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:02.659295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c98db6-dd3a-47a9-a26f-32e045c522fc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:02.659423 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c98db6-dd3a-47a9-a26f-32e045c522fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:02.677103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b7c98db6-dd3a-47a9-a26f-32e045c522fc None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:02.678281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b7c98db6-dd3a-47a9-a26f-32e045c522fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:02.685285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1215/2409] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:35:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:02.697611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f0c22-d339-4118-a9fa-45ed191fe376 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:02.708654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f0c22-d339-4118-a9fa-45ed191fe376 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:02.713880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae0ad5-1387-43d2-8b5e-6a5e0ce4992f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:02.714535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae0ad5-1387-43d2-8b5e-6a5e0ce4992f service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:02.714820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae0ad5-1387-43d2-8b5e-6a5e0ce4992f service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:02.715096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae0ad5-1387-43d2-8b5e-6a5e0ce4992f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:02.723941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae0ad5-1387-43d2-8b5e-6a5e0ce4992f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:02.725517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae0ad5-1387-43d2-8b5e-6a5e0ce4992f service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:02.734085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1195/2410] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:35:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:35:12.671743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:12.730412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:24.129465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fc48b-a53f-4e6f-b508-a7e0ddb0adf1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:24.130893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fc48b-a53f-4e6f-b508-a7e0ddb0adf1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:24.131052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fc48b-a53f-4e6f-b508-a7e0ddb0adf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:24.148912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d90fc48b-a53f-4e6f-b508-a7e0ddb0adf1 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:24.150206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d90fc48b-a53f-4e6f-b508-a7e0ddb0adf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:24.157648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1216/2411] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 17:35:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:24.178006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae0ad5-1387-43d2-8b5e-6a5e0ce4992f service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:24.188643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae0ad5-1387-43d2-8b5e-6a5e0ce4992f service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:24.194614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:24.195277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:24.195679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:24.195962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:24.204648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:24.206227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:24.215963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1196/2412] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:35:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:35:28.327083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab574da-b9a1-453a-810a-0ac582d0547b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:28.327586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab574da-b9a1-453a-810a-0ac582d0547b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:28.327729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab574da-b9a1-453a-810a-0ac582d0547b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:28.345377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6ab574da-b9a1-453a-810a-0ac582d0547b None None] MFA Rules not processed for user `7343386004e24474bc0af9126f47182b`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:28.346562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6ab574da-b9a1-453a-810a-0ac582d0547b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:28.353448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1217/2413] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:28.389841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:28.391233 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:28.447585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a4268-39ae-44fd-9f70-57efcd058b23 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:28.451114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:28.451794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:28.452073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:28.452345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:28.454085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:28.467202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:28.468563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:28.479362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1197/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:35:29.069307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab574da-b9a1-453a-810a-0ac582d0547b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:29.081109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab574da-b9a1-453a-810a-0ac582d0547b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:29.086149 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:29.087043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:29.087615 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:29.088117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:29.090048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:29.105588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:29.107427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:29.119617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1218/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:35:30.219727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:30.229438 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ba37f-3658-4bd4-b3e9-e7ecdee33671 service glance] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:30.233270 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:30.234006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:30.234632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:30.234977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:30.236898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:30.250754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:30.252306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:30.263941 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1198/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:35:33.548333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:33.549864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:33.566316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8362ee0-9a7f-456e-a3dc-0c95d9bd496a service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:33.570864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:33.571804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:33.572087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:33.572368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] PATH_INFO: `/v3/users/7fcc62869ac34381a2327be46511137b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:33.589387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fcc62869ac34381a2327be46511137b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:33.591378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:33.641755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fcc62869ac34381a2327be46511137b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:33.642400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fcc62869ac34381a2327be46511137b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:33.649964 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fcc62869ac34381a2327be46511137b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:33.662197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fcc62869ac34381a2327be46511137b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:33.662869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1cbcc13d-8103-4fc1-aecd-b2c0edb11f28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fcc62869ac34381a2327be46511137b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:33.675677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1219/2417] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:35:33 2022] DELETE /identity/v3/users/7fcc62869ac34381a2327be46511137b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:33.852346 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:33.863250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329ac0b-2887-470c-8390-c17da7de3fed service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:33.928574 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:33.929423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:33.929714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:33.929992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] PATH_INFO: `/v3/projects/90b378551d1c467e911cf01274c97d85` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:33.941473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b378551d1c467e911cf01274c97d85)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:33.944418 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:33.990698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b378551d1c467e911cf01274c97d85', 'request_id': 'req-9651450f-42ed-4442-88dc-b3c818e249d7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:33.997056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b378551d1c467e911cf01274c97d85', 'request_id': 'req-9651450f-42ed-4442-88dc-b3c818e249d7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:34.003528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b378551d1c467e911cf01274c97d85', 'request_id': 'req-9651450f-42ed-4442-88dc-b3c818e249d7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:34.011488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:34.015196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] The token cache is being invalidate because project 90b378551d1c467e911cf01274c97d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:35:34.015527 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:34.016475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9651450f-42ed-4442-88dc-b3c818e249d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:34.018345 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1199/2418] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:35:33 2022] DELETE /identity/v3/projects/90b378551d1c467e911cf01274c97d85 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:34.024072 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fd514-2406-4967-af7c-7ded8536b45a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.024363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fd514-2406-4967-af7c-7ded8536b45a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.024670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fd514-2406-4967-af7c-7ded8536b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.042215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-de9fd514-2406-4967-af7c-7ded8536b45a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:34.043433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-de9fd514-2406-4967-af7c-7ded8536b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:34.101825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1220/2419] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:34.109655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d907b-cdc7-46d0-8508-061109db009d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.109942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d907b-cdc7-46d0-8508-061109db009d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.110224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d907b-cdc7-46d0-8508-061109db009d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.128492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-360d907b-cdc7-46d0-8508-061109db009d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:34.129710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-360d907b-cdc7-46d0-8508-061109db009d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:34.138591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1200/2420] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:34.142551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fd514-2406-4967-af7c-7ded8536b45a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.152181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fd514-2406-4967-af7c-7ded8536b45a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.156623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.157263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.157542 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.157816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.158534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:35:34.158838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.160459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.172898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1221/2421] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:35:34.176355 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d907b-cdc7-46d0-8508-061109db009d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.189515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d907b-cdc7-46d0-8508-061109db009d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.194817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba0bd1-07f2-480b-bef0-09be8e3b5a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.195500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba0bd1-07f2-480b-bef0-09be8e3b5a47 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.195783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba0bd1-07f2-480b-bef0-09be8e3b5a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.196053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba0bd1-07f2-480b-bef0-09be8e3b5a47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.196735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba0bd1-07f2-480b-bef0-09be8e3b5a47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.198624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba0bd1-07f2-480b-bef0-09be8e3b5a47 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.223266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1201/2422] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:35:34 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:34.227677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.237581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3aaac-d8b6-42a5-abda-5ed83ece1a50 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.259028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:34.299531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50630cc2-554e-488f-a7fd-1ae1e614a13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.300190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630cc2-554e-488f-a7fd-1ae1e614a13a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.300469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630cc2-554e-488f-a7fd-1ae1e614a13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.300742 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630cc2-554e-488f-a7fd-1ae1e614a13a admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.301445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50630cc2-554e-488f-a7fd-1ae1e614a13a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.304137 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50630cc2-554e-488f-a7fd-1ae1e614a13a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.324037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1222/2423] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:35:34 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:34.328024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba0bd1-07f2-480b-bef0-09be8e3b5a47 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.338665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba0bd1-07f2-480b-bef0-09be8e3b5a47 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.343249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66007b2-377b-46f6-8c8b-774d28883208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.343928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66007b2-377b-46f6-8c8b-774d28883208 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.344208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66007b2-377b-46f6-8c8b-774d28883208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.344481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66007b2-377b-46f6-8c8b-774d28883208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.345205 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66007b2-377b-46f6-8c8b-774d28883208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.346779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66007b2-377b-46f6-8c8b-774d28883208 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.357395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1202/2424] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:35:34.361151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50630cc2-554e-488f-a7fd-1ae1e614a13a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.371622 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50630cc2-554e-488f-a7fd-1ae1e614a13a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.376247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96770788-a7a8-4857-b68a-21ab4018d181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.377207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770788-a7a8-4857-b68a-21ab4018d181 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.377599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770788-a7a8-4857-b68a-21ab4018d181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.377824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770788-a7a8-4857-b68a-21ab4018d181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.378934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96770788-a7a8-4857-b68a-21ab4018d181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.381633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96770788-a7a8-4857-b68a-21ab4018d181 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.391725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1223/2425] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:35:34.395729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66007b2-377b-46f6-8c8b-774d28883208 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.405545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66007b2-377b-46f6-8c8b-774d28883208 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.410092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661b5dd-6900-4460-a6f3-3ce13090a611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.410954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661b5dd-6900-4460-a6f3-3ce13090a611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.411231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661b5dd-6900-4460-a6f3-3ce13090a611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.412028 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661b5dd-6900-4460-a6f3-3ce13090a611 admin admin] PATH_INFO: `/v3/projects/240e7db1d12841c4bf539d1bf474f060/users/037a52e63d2d4920a112d5f12166bbf0/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.426471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661b5dd-6900-4460-a6f3-3ce13090a611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240e7db1d12841c4bf539d1bf474f060, user_id=037a52e63d2d4920a112d5f12166bbf0, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.431886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661b5dd-6900-4460-a6f3-3ce13090a611 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.445371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1203/2426] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:34 2022] PUT /identity/v3/projects/240e7db1d12841c4bf539d1bf474f060/users/037a52e63d2d4920a112d5f12166bbf0/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:34.456159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec02d6-6cff-42da-b320-3a1b33fc702f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.456451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec02d6-6cff-42da-b320-3a1b33fc702f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.456731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec02d6-6cff-42da-b320-3a1b33fc702f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.486620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-84ec02d6-6cff-42da-b320-3a1b33fc702f None None] MFA Rules not processed for user `037a52e63d2d4920a112d5f12166bbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:34.487861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-84ec02d6-6cff-42da-b320-3a1b33fc702f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:34.560847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1224/2427] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:34.565287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661b5dd-6900-4460-a6f3-3ce13090a611 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.640400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661b5dd-6900-4460-a6f3-3ce13090a611 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.645892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67accba3-ec54-4104-b00c-7cde787fa146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.647545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67accba3-ec54-4104-b00c-7cde787fa146 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.648197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67accba3-ec54-4104-b00c-7cde787fa146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.648482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67accba3-ec54-4104-b00c-7cde787fa146 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.649298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67accba3-ec54-4104-b00c-7cde787fa146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.651263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67accba3-ec54-4104-b00c-7cde787fa146 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.685930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1204/2428] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:35:34 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:34.689649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec02d6-6cff-42da-b320-3a1b33fc702f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.701387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec02d6-6cff-42da-b320-3a1b33fc702f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.803202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e652c4e-3073-48b5-957b-480325ab06eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.803976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652c4e-3073-48b5-957b-480325ab06eb admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.804268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652c4e-3073-48b5-957b-480325ab06eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.804544 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652c4e-3073-48b5-957b-480325ab06eb admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.805349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e652c4e-3073-48b5-957b-480325ab06eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.807493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e652c4e-3073-48b5-957b-480325ab06eb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.831855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1225/2429] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:35:34 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:34.836147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67accba3-ec54-4104-b00c-7cde787fa146 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.847199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67accba3-ec54-4104-b00c-7cde787fa146 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.852688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bdfac-cebb-4c26-b0f4-9465c2acb48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.853384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bdfac-cebb-4c26-b0f4-9465c2acb48f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.853652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bdfac-cebb-4c26-b0f4-9465c2acb48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.853931 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bdfac-cebb-4c26-b0f4-9465c2acb48f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.854712 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bdfac-cebb-4c26-b0f4-9465c2acb48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.856347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bdfac-cebb-4c26-b0f4-9465c2acb48f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.870362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1205/2430] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:35:34.872539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e652c4e-3073-48b5-957b-480325ab06eb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.885103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e652c4e-3073-48b5-957b-480325ab06eb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:34.890948 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0d912-026c-42bc-ab6e-aa7ca5222ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:34.892063 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d912-026c-42bc-ab6e-aa7ca5222ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:34.892347 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d912-026c-42bc-ab6e-aa7ca5222ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:34.892626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d912-026c-42bc-ab6e-aa7ca5222ffc admin admin] PATH_INFO: `/v3/projects/eed78f01697c400da97650b5cbdc22a7/users/aa20cb5af3cd4572af90f833b846b13a/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:34.911474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0d912-026c-42bc-ab6e-aa7ca5222ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed78f01697c400da97650b5cbdc22a7, user_id=aa20cb5af3cd4572af90f833b846b13a, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:34.914235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0d912-026c-42bc-ab6e-aa7ca5222ffc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:34.917650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:34.932109 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1226/2431] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:34 2022] PUT /identity/v3/projects/eed78f01697c400da97650b5cbdc22a7/users/aa20cb5af3cd4572af90f833b846b13a/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:35:34.936198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bdfac-cebb-4c26-b0f4-9465c2acb48f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:34.947279 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bdfac-cebb-4c26-b0f4-9465c2acb48f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:35.060757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2d5dc-f2e1-4edd-bc82-492a9ea50b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:35.061451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2d5dc-f2e1-4edd-bc82-492a9ea50b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:35.061733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2d5dc-f2e1-4edd-bc82-492a9ea50b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:35.062019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2d5dc-f2e1-4edd-bc82-492a9ea50b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:35.062763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2d5dc-f2e1-4edd-bc82-492a9ea50b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:35.069384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2d5dc-f2e1-4edd-bc82-492a9ea50b86 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:35.086609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1206/2432] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:35:35.087832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0d912-026c-42bc-ab6e-aa7ca5222ffc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:35.112223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0d912-026c-42bc-ab6e-aa7ca5222ffc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:35.121093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:35.122132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:35.122270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:35.122543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] PATH_INFO: `/v3/projects/eed78f01697c400da97650b5cbdc22a7/users/aa20cb5af3cd4572af90f833b846b13a/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:35.124730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed78f01697c400da97650b5cbdc22a7, user_id=aa20cb5af3cd4572af90f833b846b13a, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:35.127765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:35.140710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1227/2433] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:35 2022] PUT /identity/v3/projects/eed78f01697c400da97650b5cbdc22a7/users/aa20cb5af3cd4572af90f833b846b13a/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:35.151976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a458b26-1290-4efe-b744-0b509a0c1e87 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:35.152491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a458b26-1290-4efe-b744-0b509a0c1e87 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:35.152989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a458b26-1290-4efe-b744-0b509a0c1e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:35.162276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:35.187998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0a458b26-1290-4efe-b744-0b509a0c1e87 None None] MFA Rules not processed for user `aa20cb5af3cd4572af90f833b846b13a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:35.189228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0a458b26-1290-4efe-b744-0b509a0c1e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:35.264459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1207/2434] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:35.276660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:35.278351 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:35.342366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166e41d-4c01-400c-b54c-59bbcffc3e53 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:35.346169 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcdf5a-ffb4-4641-9616-b00adc016f0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:35.346859 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcdf5a-ffb4-4641-9616-b00adc016f0c service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:35.347146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcdf5a-ffb4-4641-9616-b00adc016f0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:35.347448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcdf5a-ffb4-4641-9616-b00adc016f0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:35.356887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabcdf5a-ffb4-4641-9616-b00adc016f0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:35.358792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabcdf5a-ffb4-4641-9616-b00adc016f0c service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:35.370846 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1228/2435] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:35 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:35:35.446330 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a458b26-1290-4efe-b744-0b509a0c1e87 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:35.456005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a458b26-1290-4efe-b744-0b509a0c1e87 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:35.460401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b4ed8-6321-449b-b8f9-894b2e347d2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:35.461035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b4ed8-6321-449b-b8f9-894b2e347d2d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:35.461318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b4ed8-6321-449b-b8f9-894b2e347d2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:35.461596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b4ed8-6321-449b-b8f9-894b2e347d2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:35.517218 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b4ed8-6321-449b-b8f9-894b2e347d2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:35.518639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b4ed8-6321-449b-b8f9-894b2e347d2d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:35.528131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1208/2436] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:35:45.366396 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:45.524158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=60 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:46.808415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcdf5a-ffb4-4641-9616-b00adc016f0c service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:46.869339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcdf5a-ffb4-4641-9616-b00adc016f0c service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:46.873783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:46.874634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:46.874909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:46.875177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] PATH_INFO: `/v3/users/037a52e63d2d4920a112d5f12166bbf0` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:46.876364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=037a52e63d2d4920a112d5f12166bbf0)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:46.878285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:46.922896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '037a52e63d2d4920a112d5f12166bbf0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:46.923490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '037a52e63d2d4920a112d5f12166bbf0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:46.930987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '037a52e63d2d4920a112d5f12166bbf0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:46.943749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '037a52e63d2d4920a112d5f12166bbf0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:46.944394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '037a52e63d2d4920a112d5f12166bbf0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:46.956834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1229/2437] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:35:46 2022] DELETE /identity/v3/users/037a52e63d2d4920a112d5f12166bbf0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:46.961421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be82b-96b0-4850-b786-ce52596dfbad None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:46.961928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be82b-96b0-4850-b786-ce52596dfbad None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:46.962052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be82b-96b0-4850-b786-ce52596dfbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:46.979887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c12be82b-96b0-4850-b786-ce52596dfbad None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:46.981065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c12be82b-96b0-4850-b786-ce52596dfbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:47.041907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1209/2438] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:47.052554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:47.054421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:47.111476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7ec2-02be-406a-993b-53250ca53a9d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:47.115777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d41a1c-2a14-4c49-a1cc-e893022cce83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:47.116418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d41a1c-2a14-4c49-a1cc-e893022cce83 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:47.116729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d41a1c-2a14-4c49-a1cc-e893022cce83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:47.117007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d41a1c-2a14-4c49-a1cc-e893022cce83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:47.126062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d41a1c-2a14-4c49-a1cc-e893022cce83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:47.127530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d41a1c-2a14-4c49-a1cc-e893022cce83 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:47.137803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1230/2439] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:35:47.400927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be82b-96b0-4850-b786-ce52596dfbad None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:47.402506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c12be82b-96b0-4850-b786-ce52596dfbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:47.415977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be82b-96b0-4850-b786-ce52596dfbad None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:47.420565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:47.421435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:47.421717 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:47.421989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] PATH_INFO: `/v3/projects/240e7db1d12841c4bf539d1bf474f060` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:47.423238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=240e7db1d12841c4bf539d1bf474f060)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:47.425215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:47.471944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240e7db1d12841c4bf539d1bf474f060', 'request_id': 'req-317a7fff-6339-427a-9504-9c33e6b75a7e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.479733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240e7db1d12841c4bf539d1bf474f060', 'request_id': 'req-317a7fff-6339-427a-9504-9c33e6b75a7e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.485714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240e7db1d12841c4bf539d1bf474f060', 'request_id': 'req-317a7fff-6339-427a-9504-9c33e6b75a7e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.496570 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] The token cache is being invalidate because project 240e7db1d12841c4bf539d1bf474f060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:35:47.496859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.497488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.499015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1210/2440] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:35:47 2022] DELETE /identity/v3/projects/240e7db1d12841c4bf539d1bf474f060 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:47.502911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d41a1c-2a14-4c49-a1cc-e893022cce83 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:47.512600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d41a1c-2a14-4c49-a1cc-e893022cce83 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:47.530941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:47.570912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:47.571818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:47.572102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:47.572378 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] PATH_INFO: `/v3/users/aa20cb5af3cd4572af90f833b846b13a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:47.573559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa20cb5af3cd4572af90f833b846b13a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:47.575545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:47.622937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa20cb5af3cd4572af90f833b846b13a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.623602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa20cb5af3cd4572af90f833b846b13a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.630834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa20cb5af3cd4572af90f833b846b13a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.642536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa20cb5af3cd4572af90f833b846b13a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.643168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e8f59783-fc97-442c-acdc-97588bde83ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa20cb5af3cd4572af90f833b846b13a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:47.656247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1231/2441] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:35:47 2022] DELETE /identity/v3/users/aa20cb5af3cd4572af90f833b846b13a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:47.927608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:47.929070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:47.941558 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a7fff-6339-427a-9504-9c33e6b75a7e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:47.998200 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:47.999067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:47.999362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:47.999645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] PATH_INFO: `/v3/projects/eed78f01697c400da97650b5cbdc22a7` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.000818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed78f01697c400da97650b5cbdc22a7)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.002717 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.033705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:48.051186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed78f01697c400da97650b5cbdc22a7', 'request_id': 'req-3201e20b-4bea-4789-927a-6c8dbef42f4e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:48.057565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed78f01697c400da97650b5cbdc22a7', 'request_id': 'req-3201e20b-4bea-4789-927a-6c8dbef42f4e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:48.063601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed78f01697c400da97650b5cbdc22a7', 'request_id': 'req-3201e20b-4bea-4789-927a-6c8dbef42f4e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:48.075905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] The token cache is being invalidate because project eed78f01697c400da97650b5cbdc22a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:35:48.076220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:48.076925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3201e20b-4bea-4789-927a-6c8dbef42f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:48.078580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1211/2442] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:35:47 2022] DELETE /identity/v3/projects/eed78f01697c400da97650b5cbdc22a7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:35:48.085660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ae57c-eb1c-4ef1-8d2f-6f93fdc4ee90 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.086180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ae57c-eb1c-4ef1-8d2f-6f93fdc4ee90 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.086302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ae57c-eb1c-4ef1-8d2f-6f93fdc4ee90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.104661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-917ae57c-eb1c-4ef1-8d2f-6f93fdc4ee90 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:48.105924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-917ae57c-eb1c-4ef1-8d2f-6f93fdc4ee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:48.163863 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1232/2443] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:48.171730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b13d-66b4-4a0f-aa48-2d0e2033f620 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.172024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b13d-66b4-4a0f-aa48-2d0e2033f620 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.172304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b13d-66b4-4a0f-aa48-2d0e2033f620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.190124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b518b13d-66b4-4a0f-aa48-2d0e2033f620 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:48.191316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b518b13d-66b4-4a0f-aa48-2d0e2033f620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:48.200141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1212/2444] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:48.204227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ae57c-eb1c-4ef1-8d2f-6f93fdc4ee90 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.214502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ae57c-eb1c-4ef1-8d2f-6f93fdc4ee90 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.218992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.219665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.219962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.220238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.220983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:35:48.221310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.222898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.234384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1233/2445] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:35:48.238523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518b13d-66b4-4a0f-aa48-2d0e2033f620 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.247837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518b13d-66b4-4a0f-aa48-2d0e2033f620 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.252531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bafa0-b9a1-49f0-9ba2-96c0d1e64a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.253176 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bafa0-b9a1-49f0-9ba2-96c0d1e64a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.253455 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bafa0-b9a1-49f0-9ba2-96c0d1e64a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.253730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bafa0-b9a1-49f0-9ba2-96c0d1e64a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.255196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bafa0-b9a1-49f0-9ba2-96c0d1e64a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.257246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bafa0-b9a1-49f0-9ba2-96c0d1e64a46 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.282129 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1213/2446] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:35:48 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:48.286329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.296200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec2dfe-2b7f-4be8-bfa7-93adb6864896 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.322734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:48.355038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd28784-51b8-41a6-9cb5-38b57ac2bc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.355721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd28784-51b8-41a6-9cb5-38b57ac2bc6e admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.356001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd28784-51b8-41a6-9cb5-38b57ac2bc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.356276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd28784-51b8-41a6-9cb5-38b57ac2bc6e admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.356977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd28784-51b8-41a6-9cb5-38b57ac2bc6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.358900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd28784-51b8-41a6-9cb5-38b57ac2bc6e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.380677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1234/2447] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:35:48 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:48.384520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bafa0-b9a1-49f0-9ba2-96c0d1e64a46 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.394945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bafa0-b9a1-49f0-9ba2-96c0d1e64a46 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.399617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b15fc-0bff-4969-9e83-18ea7d363b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.400280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b15fc-0bff-4969-9e83-18ea7d363b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.400566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b15fc-0bff-4969-9e83-18ea7d363b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.400848 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b15fc-0bff-4969-9e83-18ea7d363b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.401601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b15fc-0bff-4969-9e83-18ea7d363b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.403188 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b15fc-0bff-4969-9e83-18ea7d363b20 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.413659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1214/2448] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:35:48.417343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd28784-51b8-41a6-9cb5-38b57ac2bc6e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.427871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd28784-51b8-41a6-9cb5-38b57ac2bc6e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.432401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f138-f78d-4225-b8a4-f822361f678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.433054 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f138-f78d-4225-b8a4-f822361f678c admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.433331 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f138-f78d-4225-b8a4-f822361f678c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.433599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f138-f78d-4225-b8a4-f822361f678c admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.434313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5f138-f78d-4225-b8a4-f822361f678c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.435937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5f138-f78d-4225-b8a4-f822361f678c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.446280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1235/2449] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:35:48.450212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b15fc-0bff-4969-9e83-18ea7d363b20 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.460524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b15fc-0bff-4969-9e83-18ea7d363b20 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.465081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e32f9-b78a-4c8b-b67d-e00a4465ab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.465948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e32f9-b78a-4c8b-b67d-e00a4465ab1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.466226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e32f9-b78a-4c8b-b67d-e00a4465ab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.466504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e32f9-b78a-4c8b-b67d-e00a4465ab1f admin admin] PATH_INFO: `/v3/projects/2a5c6f46d1e34a13a46564243d6f750c/users/04269df8da37403dbea95032a47b8352/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.481136 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e32f9-b78a-4c8b-b67d-e00a4465ab1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5c6f46d1e34a13a46564243d6f750c, user_id=04269df8da37403dbea95032a47b8352, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.483798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e32f9-b78a-4c8b-b67d-e00a4465ab1f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.497084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1215/2450] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:48 2022] PUT /identity/v3/projects/2a5c6f46d1e34a13a46564243d6f750c/users/04269df8da37403dbea95032a47b8352/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:48.509387 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4af56-be1d-4b79-baea-07880e1419f3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.509677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4af56-be1d-4b79-baea-07880e1419f3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.509956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4af56-be1d-4b79-baea-07880e1419f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.536113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-49d4af56-be1d-4b79-baea-07880e1419f3 None None] MFA Rules not processed for user `04269df8da37403dbea95032a47b8352`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:48.537438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-49d4af56-be1d-4b79-baea-07880e1419f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:48.600838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1236/2451] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:48.605226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e32f9-b78a-4c8b-b67d-e00a4465ab1f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.667043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e32f9-b78a-4c8b-b67d-e00a4465ab1f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.671616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733b1da-5651-45ae-a4d8-6829912553aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.672267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733b1da-5651-45ae-a4d8-6829912553aa admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.672545 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733b1da-5651-45ae-a4d8-6829912553aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.672817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733b1da-5651-45ae-a4d8-6829912553aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.673498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733b1da-5651-45ae-a4d8-6829912553aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.675407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733b1da-5651-45ae-a4d8-6829912553aa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.700738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1216/2452] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:35:48 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:48.704721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4af56-be1d-4b79-baea-07880e1419f3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.715429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4af56-be1d-4b79-baea-07880e1419f3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.773091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1140-90ea-4a0a-af67-9a0b81fe7378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.773754 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1140-90ea-4a0a-af67-9a0b81fe7378 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.774036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1140-90ea-4a0a-af67-9a0b81fe7378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.774314 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1140-90ea-4a0a-af67-9a0b81fe7378 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.775008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b1140-90ea-4a0a-af67-9a0b81fe7378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.776968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b1140-90ea-4a0a-af67-9a0b81fe7378 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.799596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1237/2453] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:35:48 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:35:48.800795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733b1da-5651-45ae-a4d8-6829912553aa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.811855 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733b1da-5651-45ae-a4d8-6829912553aa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.816463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d2afd-0a4c-4ef3-bb2a-a06c66dec9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.817222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d2afd-0a4c-4ef3-bb2a-a06c66dec9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.817542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d2afd-0a4c-4ef3-bb2a-a06c66dec9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.817824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d2afd-0a4c-4ef3-bb2a-a06c66dec9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.818601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d2afd-0a4c-4ef3-bb2a-a06c66dec9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.820318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d2afd-0a4c-4ef3-bb2a-a06c66dec9fc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.831033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1217/2454] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:35:48.835267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1140-90ea-4a0a-af67-9a0b81fe7378 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.844915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1140-90ea-4a0a-af67-9a0b81fe7378 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.850266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ca89d-963f-4e4e-b404-d2ea281f386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.851132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ca89d-963f-4e4e-b404-d2ea281f386b admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.851442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ca89d-963f-4e4e-b404-d2ea281f386b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.851717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ca89d-963f-4e4e-b404-d2ea281f386b admin admin] PATH_INFO: `/v3/projects/daf0bf578df44e04bbfe707bfba6a033/users/00b0530f256b48ba8e06f01abda150ad/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.867808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ca89d-963f-4e4e-b404-d2ea281f386b admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf0bf578df44e04bbfe707bfba6a033, user_id=00b0530f256b48ba8e06f01abda150ad, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.870437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ca89d-963f-4e4e-b404-d2ea281f386b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.883045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1238/2455] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:48 2022] PUT /identity/v3/projects/daf0bf578df44e04bbfe707bfba6a033/users/00b0530f256b48ba8e06f01abda150ad/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:48.886966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d2afd-0a4c-4ef3-bb2a-a06c66dec9fc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.897313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d2afd-0a4c-4ef3-bb2a-a06c66dec9fc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.917313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:48.954228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84d0d-2e90-47d4-b875-c83e45e2f652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.954887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84d0d-2e90-47d4-b875-c83e45e2f652 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.955169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84d0d-2e90-47d4-b875-c83e45e2f652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.955486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84d0d-2e90-47d4-b875-c83e45e2f652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.956209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84d0d-2e90-47d4-b875-c83e45e2f652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.958142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84d0d-2e90-47d4-b875-c83e45e2f652 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:48.968415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1218/2456] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:35:48.972518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ca89d-963f-4e4e-b404-d2ea281f386b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:48.978730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:48.985025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ca89d-963f-4e4e-b404-d2ea281f386b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:48.990925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:48.991849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:48.992127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:48.992396 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] PATH_INFO: `/v3/projects/daf0bf578df44e04bbfe707bfba6a033/users/00b0530f256b48ba8e06f01abda150ad/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:48.994344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf0bf578df44e04bbfe707bfba6a033, user_id=00b0530f256b48ba8e06f01abda150ad, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:48.997002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:49.007596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1239/2457] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:48 2022] PUT /identity/v3/projects/daf0bf578df44e04bbfe707bfba6a033/users/00b0530f256b48ba8e06f01abda150ad/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:49.017632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe061f1-2bb6-438f-babe-ceb77245039d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:49.018079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe061f1-2bb6-438f-babe-ceb77245039d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:49.018196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe061f1-2bb6-438f-babe-ceb77245039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:49.044902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-afe061f1-2bb6-438f-babe-ceb77245039d None None] MFA Rules not processed for user `00b0530f256b48ba8e06f01abda150ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:49.046721 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-afe061f1-2bb6-438f-babe-ceb77245039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:49.113945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1219/2458] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:49.124708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:49.126482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:49.182963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496f659-c61f-43b7-941c-d8baad4abc00 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:49.186594 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee6569-9009-4e61-afe1-cc1c8dec56d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:49.187227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee6569-9009-4e61-afe1-cc1c8dec56d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:49.187535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee6569-9009-4e61-afe1-cc1c8dec56d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:49.187812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee6569-9009-4e61-afe1-cc1c8dec56d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:49.196660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee6569-9009-4e61-afe1-cc1c8dec56d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:49.198077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee6569-9009-4e61-afe1-cc1c8dec56d2 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:49.207319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1240/2459] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:49 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:35:50.494526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe061f1-2bb6-438f-babe-ceb77245039d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:50.559860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe061f1-2bb6-438f-babe-ceb77245039d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:50.564579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:50.565472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:50.565749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:50.566020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] PATH_INFO: `/v3/users/04269df8da37403dbea95032a47b8352` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:50.568260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] RBAC: Authorizing `identity:delete_user(user_id=04269df8da37403dbea95032a47b8352)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:50.570155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:50.617065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04269df8da37403dbea95032a47b8352'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:50.618748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:50.625041 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04269df8da37403dbea95032a47b8352'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:50.638208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04269df8da37403dbea95032a47b8352'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:50.638834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04269df8da37403dbea95032a47b8352'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:50.645879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04269df8da37403dbea95032a47b8352'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:50.651910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1220/2460] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:35:50 2022] DELETE /identity/v3/users/04269df8da37403dbea95032a47b8352 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:50.656367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d77b71-7b60-4d10-bc55-37920028e0a7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:50.656845 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d77b71-7b60-4d10-bc55-37920028e0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:50.656943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d77b71-7b60-4d10-bc55-37920028e0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:50.677236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d9d77b71-7b60-4d10-bc55-37920028e0a7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:50.678380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d9d77b71-7b60-4d10-bc55-37920028e0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:50.748792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1241/2461] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:35:50.758930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:50.760358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:50.817785 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d2fbd-cd32-4d21-9223-9417c40ec48e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:50.821446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679d198-5ac7-4360-a4f5-6f59a05c2e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:50.822090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d198-5ac7-4360-a4f5-6f59a05c2e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:50.822374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d198-5ac7-4360-a4f5-6f59a05c2e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:50.822650 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d198-5ac7-4360-a4f5-6f59a05c2e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:50.831626 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679d198-5ac7-4360-a4f5-6f59a05c2e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:50.833091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679d198-5ac7-4360-a4f5-6f59a05c2e5b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:50.842350 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1221/2462] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:35:51.118455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d77b71-7b60-4d10-bc55-37920028e0a7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:51.119982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d9d77b71-7b60-4d10-bc55-37920028e0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:51.132851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d77b71-7b60-4d10-bc55-37920028e0a7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:51.137527 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:51.138422 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:51.138703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:51.138969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] PATH_INFO: `/v3/projects/2a5c6f46d1e34a13a46564243d6f750c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:51.140194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5c6f46d1e34a13a46564243d6f750c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:51.142112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:51.188730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5c6f46d1e34a13a46564243d6f750c', 'request_id': 'req-61199b0f-5ba9-45e7-80d3-355a74b039de'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.195389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5c6f46d1e34a13a46564243d6f750c', 'request_id': 'req-61199b0f-5ba9-45e7-80d3-355a74b039de'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.201036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5c6f46d1e34a13a46564243d6f750c', 'request_id': 'req-61199b0f-5ba9-45e7-80d3-355a74b039de'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.212998 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] The token cache is being invalidate because project 2a5c6f46d1e34a13a46564243d6f750c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:35:51.213279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.213887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.215417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1242/2463] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:35:51 2022] DELETE /identity/v3/projects/2a5c6f46d1e34a13a46564243d6f750c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:51.219240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679d198-5ac7-4360-a4f5-6f59a05c2e5b service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:51.229419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679d198-5ac7-4360-a4f5-6f59a05c2e5b service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:51.287578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:51.288453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:51.288731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:51.289004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] PATH_INFO: `/v3/users/00b0530f256b48ba8e06f01abda150ad` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:51.290167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b0530f256b48ba8e06f01abda150ad)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:51.292108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:51.337421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b0530f256b48ba8e06f01abda150ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.344652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b0530f256b48ba8e06f01abda150ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.356379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:51.357523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b0530f256b48ba8e06f01abda150ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.358301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b0530f256b48ba8e06f01abda150ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.367331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c52acb2c-dbd7-4ff4-82bd-d3af9d525b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b0530f256b48ba8e06f01abda150ad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.373311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1222/2464] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:35:51 2022] DELETE /identity/v3/users/00b0530f256b48ba8e06f01abda150ad => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:51.640704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:51.642238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:51.658359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199b0f-5ba9-45e7-80d3-355a74b039de admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:51.672124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:51.721680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:51.722487 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:51.722769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:51.723045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] PATH_INFO: `/v3/projects/daf0bf578df44e04bbfe707bfba6a033` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:51.724983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf0bf578df44e04bbfe707bfba6a033)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:51.726910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:51.773015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf0bf578df44e04bbfe707bfba6a033', 'request_id': 'req-10b37699-f46e-4be0-b984-725893bc6c45'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.779058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf0bf578df44e04bbfe707bfba6a033', 'request_id': 'req-10b37699-f46e-4be0-b984-725893bc6c45'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.784460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf0bf578df44e04bbfe707bfba6a033', 'request_id': 'req-10b37699-f46e-4be0-b984-725893bc6c45'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.797693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] The token cache is being invalidate because project daf0bf578df44e04bbfe707bfba6a033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:35:51.797979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.798599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10b37699-f46e-4be0-b984-725893bc6c45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:35:51.800228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1243/2465] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:35:51 2022] DELETE /identity/v3/projects/daf0bf578df44e04bbfe707bfba6a033 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:35:51.805880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0df7c-d5ad-469d-9378-27d1f64ee029 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:51.806360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0df7c-d5ad-469d-9378-27d1f64ee029 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:51.806472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0df7c-d5ad-469d-9378-27d1f64ee029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:51.824706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-79c0df7c-d5ad-469d-9378-27d1f64ee029 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:51.825916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-79c0df7c-d5ad-469d-9378-27d1f64ee029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:51.883804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1223/2466] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:51.892451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5fe2-1dbc-4ba7-abba-517abb709777 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:51.892740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5fe2-1dbc-4ba7-abba-517abb709777 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:51.893016 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5fe2-1dbc-4ba7-abba-517abb709777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:51.911412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e64a5fe2-1dbc-4ba7-abba-517abb709777 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:51.912598 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e64a5fe2-1dbc-4ba7-abba-517abb709777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:51.920421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1244/2467] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:51.924575 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0df7c-d5ad-469d-9378-27d1f64ee029 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:51.934749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0df7c-d5ad-469d-9378-27d1f64ee029 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:51.939277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:51.939955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:51.940243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:51.940522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:51.941253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:35:51.941596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:51.943284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:51.954680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1224/2468] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:35:51.958709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a5fe2-1dbc-4ba7-abba-517abb709777 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:51.967762 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a5fe2-1dbc-4ba7-abba-517abb709777 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:51.972392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd7705-c83c-4a88-b62c-6ef97e250638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:51.972979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd7705-c83c-4a88-b62c-6ef97e250638 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:51.973812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd7705-c83c-4a88-b62c-6ef97e250638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:51.974240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd7705-c83c-4a88-b62c-6ef97e250638 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:51.974927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd7705-c83c-4a88-b62c-6ef97e250638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:51.976872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd7705-c83c-4a88-b62c-6ef97e250638 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.002675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1245/2469] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:35:51 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:52.007197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.017186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17970435-4492-41fe-923c-4306ce86659b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.072975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be600233-3066-4a3c-a1c0-ad1c836c1594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.073645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be600233-3066-4a3c-a1c0-ad1c836c1594 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.073929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be600233-3066-4a3c-a1c0-ad1c836c1594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.074207 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be600233-3066-4a3c-a1c0-ad1c836c1594 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.074900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be600233-3066-4a3c-a1c0-ad1c836c1594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.076856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be600233-3066-4a3c-a1c0-ad1c836c1594 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.096779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1225/2470] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:35:52 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:52.101607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd7705-c83c-4a88-b62c-6ef97e250638 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.111682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd7705-c83c-4a88-b62c-6ef97e250638 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.116336 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c4cb-1c59-421a-85ce-be5d8b75c6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.116969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c4cb-1c59-421a-85ce-be5d8b75c6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.117250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c4cb-1c59-421a-85ce-be5d8b75c6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.117526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c4cb-1c59-421a-85ce-be5d8b75c6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.118251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c4cb-1c59-421a-85ce-be5d8b75c6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.119886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c4cb-1c59-421a-85ce-be5d8b75c6e8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.130359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1246/2471] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:35:52.134062 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be600233-3066-4a3c-a1c0-ad1c836c1594 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.143682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be600233-3066-4a3c-a1c0-ad1c836c1594 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.148805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90646-25c2-40ce-a8ec-813677b66d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.149454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90646-25c2-40ce-a8ec-813677b66d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.149736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90646-25c2-40ce-a8ec-813677b66d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.150012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90646-25c2-40ce-a8ec-813677b66d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.150734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d90646-25c2-40ce-a8ec-813677b66d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.152344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d90646-25c2-40ce-a8ec-813677b66d14 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.162680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1226/2472] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:35:52.166342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c4cb-1c59-421a-85ce-be5d8b75c6e8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.175924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c4cb-1c59-421a-85ce-be5d8b75c6e8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.180565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceae8ac-6d97-4e26-8e84-15808fc22fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.181429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceae8ac-6d97-4e26-8e84-15808fc22fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.181710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceae8ac-6d97-4e26-8e84-15808fc22fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.181984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceae8ac-6d97-4e26-8e84-15808fc22fbc admin admin] PATH_INFO: `/v3/projects/9e8db3a27a3741c4a8c809f8485bfc14/users/881e343e826243afbef410189bb11f9c/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.196313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceae8ac-6d97-4e26-8e84-15808fc22fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8db3a27a3741c4a8c809f8485bfc14, user_id=881e343e826243afbef410189bb11f9c, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.198955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceae8ac-6d97-4e26-8e84-15808fc22fbc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.212339 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1247/2473] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:52 2022] PUT /identity/v3/projects/9e8db3a27a3741c4a8c809f8485bfc14/users/881e343e826243afbef410189bb11f9c/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:35:52.222548 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad41811-9603-4e9e-a6c9-d77520f3ae51 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.222789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad41811-9603-4e9e-a6c9-d77520f3ae51 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.223071 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad41811-9603-4e9e-a6c9-d77520f3ae51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.249375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-dad41811-9603-4e9e-a6c9-d77520f3ae51 None None] MFA Rules not processed for user `881e343e826243afbef410189bb11f9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:52.250580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dad41811-9603-4e9e-a6c9-d77520f3ae51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:52.291531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:52.314731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1227/2474] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:52.319292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceae8ac-6d97-4e26-8e84-15808fc22fbc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.363482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:52.382178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceae8ac-6d97-4e26-8e84-15808fc22fbc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.386763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a560223-5b84-4c1f-ba81-d2404aa6f343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.387445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a560223-5b84-4c1f-ba81-d2404aa6f343 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.387730 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a560223-5b84-4c1f-ba81-d2404aa6f343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.388005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a560223-5b84-4c1f-ba81-d2404aa6f343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.388689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a560223-5b84-4c1f-ba81-d2404aa6f343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.390596 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a560223-5b84-4c1f-ba81-d2404aa6f343 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.415494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1248/2475] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:35:52 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:35:52.419243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad41811-9603-4e9e-a6c9-d77520f3ae51 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.429728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad41811-9603-4e9e-a6c9-d77520f3ae51 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.487116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d4d47-a16f-439a-adaf-6b1eeaf8ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.487751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d4d47-a16f-439a-adaf-6b1eeaf8ca64 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.488033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d4d47-a16f-439a-adaf-6b1eeaf8ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.488311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d4d47-a16f-439a-adaf-6b1eeaf8ca64 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.489075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d4d47-a16f-439a-adaf-6b1eeaf8ca64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.491100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d4d47-a16f-439a-adaf-6b1eeaf8ca64 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.510995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1228/2476] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:35:52 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:35:52.514962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a560223-5b84-4c1f-ba81-d2404aa6f343 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.526408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a560223-5b84-4c1f-ba81-d2404aa6f343 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.531143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa071a-a89b-404c-b821-f68cd2b0f4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.531828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa071a-a89b-404c-b821-f68cd2b0f4ae admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.532116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa071a-a89b-404c-b821-f68cd2b0f4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.532388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa071a-a89b-404c-b821-f68cd2b0f4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.533132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa071a-a89b-404c-b821-f68cd2b0f4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.534721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa071a-a89b-404c-b821-f68cd2b0f4ae admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.545743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1249/2477] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:35:52.550337 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d4d47-a16f-439a-adaf-6b1eeaf8ca64 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.560521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d4d47-a16f-439a-adaf-6b1eeaf8ca64 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.565211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0743163-1d98-4b9e-9bde-038632b3a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.566116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0743163-1d98-4b9e-9bde-038632b3a590 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.566402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0743163-1d98-4b9e-9bde-038632b3a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.566680 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0743163-1d98-4b9e-9bde-038632b3a590 admin admin] PATH_INFO: `/v3/projects/eb10687f73454559a363962e5e309735/users/db2ec26fa8054ff29f4e84b4c7b89008/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.585436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0743163-1d98-4b9e-9bde-038632b3a590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb10687f73454559a363962e5e309735, user_id=db2ec26fa8054ff29f4e84b4c7b89008, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.588181 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0743163-1d98-4b9e-9bde-038632b3a590 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.600805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1229/2478] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:52 2022] PUT /identity/v3/projects/eb10687f73454559a363962e5e309735/users/db2ec26fa8054ff29f4e84b4c7b89008/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:52.604713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa071a-a89b-404c-b821-f68cd2b0f4ae admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.614389 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa071a-a89b-404c-b821-f68cd2b0f4ae admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.670605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9251a08-d866-4085-8905-fdc28e93c239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.671293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9251a08-d866-4085-8905-fdc28e93c239 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.671602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9251a08-d866-4085-8905-fdc28e93c239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.671877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9251a08-d866-4085-8905-fdc28e93c239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.672602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9251a08-d866-4085-8905-fdc28e93c239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.674209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9251a08-d866-4085-8905-fdc28e93c239 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.684156 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1250/2479] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:35:52.688202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0743163-1d98-4b9e-9bde-038632b3a590 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.699468 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0743163-1d98-4b9e-9bde-038632b3a590 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.704090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.704974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.705255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.705528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] PATH_INFO: `/v3/projects/eb10687f73454559a363962e5e309735/users/db2ec26fa8054ff29f4e84b4c7b89008/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.708189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb10687f73454559a363962e5e309735, user_id=db2ec26fa8054ff29f4e84b4c7b89008, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.710826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.722474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1230/2480] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:35:52 2022] PUT /identity/v3/projects/eb10687f73454559a363962e5e309735/users/db2ec26fa8054ff29f4e84b4c7b89008/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:35:52.731744 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52be63f-61cf-4326-9f25-fcace17676ca None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.732043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52be63f-61cf-4326-9f25-fcace17676ca None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.732313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52be63f-61cf-4326-9f25-fcace17676ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.758720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e52be63f-61cf-4326-9f25-fcace17676ca None None] MFA Rules not processed for user `db2ec26fa8054ff29f4e84b4c7b89008`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:35:52.759937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e52be63f-61cf-4326-9f25-fcace17676ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:52.828446 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1251/2481] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:35:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:35:52.839251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:52.841080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:52.901707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbb27f-9795-4a02-8753-300bfcc83b31 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:52.905388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:52.906039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:52.906325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:52.906609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:52.952740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:35:52.963466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:52.964910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:52.973992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1231/2482] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:35:58.612743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52be63f-61cf-4326-9f25-fcace17676ca None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:35:58.615135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e52be63f-61cf-4326-9f25-fcace17676ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:35:58.626702 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52be63f-61cf-4326-9f25-fcace17676ca None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:35:58.631302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:35:58.632029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:35:58.632310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:35:58.632585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:35:58.641408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:35:58.642818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:35:58.652456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1252/2483] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:35:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:02.969667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:08.648379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:24.609885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:24.611288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:24.668523 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd920-d912-4fb1-8e33-2a5eaaac5bab service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:24.672074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:24.672715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:24.672993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:24.673260 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:24.674980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:24.732563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:24.733991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:24.743469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1232/2484] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:25.595588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:25.597069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:25.678417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff4599-7c3e-4f51-9fa5-f5afe027e90c service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:25.684517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:25.685201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:25.685476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:25.686040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:25.687718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:25.701059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:25.703052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:25.716175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1253/2485] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:27.030616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:27.040344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c99b21-cb60-4f32-81cf-a8467688ded9 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:27.044140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf76f2-2651-4a99-9814-191a6e2819f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:27.044901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf76f2-2651-4a99-9814-191a6e2819f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:27.045192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf76f2-2651-4a99-9814-191a6e2819f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:27.045464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf76f2-2651-4a99-9814-191a6e2819f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:27.047243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-99cf76f2-2651-4a99-9814-191a6e2819f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:27.057423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf76f2-2651-4a99-9814-191a6e2819f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:27.058925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf76f2-2651-4a99-9814-191a6e2819f2 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:27.068106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1233/2486] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:27.549975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:27.613229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1891a63-6b4e-43c2-bdc5-1b496d9ba562 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:27.618416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:27.619293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:27.619597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:27.619872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] PATH_INFO: `/v3/users/881e343e826243afbef410189bb11f9c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:27.621087 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=881e343e826243afbef410189bb11f9c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:27.623295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:27.669637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '881e343e826243afbef410189bb11f9c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:27.670283 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '881e343e826243afbef410189bb11f9c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:27.678332 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '881e343e826243afbef410189bb11f9c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:27.689849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '881e343e826243afbef410189bb11f9c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:27.690485 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '881e343e826243afbef410189bb11f9c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:27.703043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1254/2487] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:27 2022] DELETE /identity/v3/users/881e343e826243afbef410189bb11f9c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:27.707732 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959647-28bf-46cf-9c04-dfcf5da3c2de None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:27.708040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959647-28bf-46cf-9c04-dfcf5da3c2de None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:27.708520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959647-28bf-46cf-9c04-dfcf5da3c2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:27.727471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-54959647-28bf-46cf-9c04-dfcf5da3c2de None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:27.728746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-54959647-28bf-46cf-9c04-dfcf5da3c2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:27.799135 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1234/2488] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:36:27.809442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:27.810893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:27.817440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:27.881591 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093110e-169f-4582-95c4-a4b69f150ce7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:27.885976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e745b-b199-4ced-a396-5c2af70cdd64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:27.886627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e745b-b199-4ced-a396-5c2af70cdd64 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:27.886912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e745b-b199-4ced-a396-5c2af70cdd64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:27.887186 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e745b-b199-4ced-a396-5c2af70cdd64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:27.897895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e745b-b199-4ced-a396-5c2af70cdd64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:27.899502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e745b-b199-4ced-a396-5c2af70cdd64 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:27.909778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1255/2489] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:28.457758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54959647-28bf-46cf-9c04-dfcf5da3c2de None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:28.459972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-54959647-28bf-46cf-9c04-dfcf5da3c2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:28.472708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54959647-28bf-46cf-9c04-dfcf5da3c2de None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:28.478789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:28.479796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:28.480349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:28.480686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] PATH_INFO: `/v3/projects/9e8db3a27a3741c4a8c809f8485bfc14` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:28.481946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e8db3a27a3741c4a8c809f8485bfc14)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:28.483888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:28.524546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:28.535446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8db3a27a3741c4a8c809f8485bfc14', 'request_id': 'req-68b4290b-1070-447b-b28d-d5105c617013'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.543440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8db3a27a3741c4a8c809f8485bfc14', 'request_id': 'req-68b4290b-1070-447b-b28d-d5105c617013'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.549790 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8db3a27a3741c4a8c809f8485bfc14', 'request_id': 'req-68b4290b-1070-447b-b28d-d5105c617013'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.560834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] The token cache is being invalidate because project 9e8db3a27a3741c4a8c809f8485bfc14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:28.561125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.561751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.563305 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1235/2490] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:28 2022] DELETE /identity/v3/projects/9e8db3a27a3741c4a8c809f8485bfc14 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:28.567313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e745b-b199-4ced-a396-5c2af70cdd64 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:28.577135 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e745b-b199-4ced-a396-5c2af70cdd64 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:28.636982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:28.637914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:28.638196 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:28.638469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] PATH_INFO: `/v3/users/db2ec26fa8054ff29f4e84b4c7b89008` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:28.639772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2ec26fa8054ff29f4e84b4c7b89008)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:28.641737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:28.696166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2ec26fa8054ff29f4e84b4c7b89008'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.696834 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2ec26fa8054ff29f4e84b4c7b89008'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.704452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2ec26fa8054ff29f4e84b4c7b89008'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.721548 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2ec26fa8054ff29f4e84b4c7b89008'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.722709 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-19c5fd9d-024c-4c69-a673-969ebc34bcda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2ec26fa8054ff29f4e84b4c7b89008'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:28.742953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1256/2491] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:28 2022] DELETE /identity/v3/users/db2ec26fa8054ff29f4e84b4c7b89008 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:29.028748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.030311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:29.045610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4290b-1070-447b-b28d-d5105c617013 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.101874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.102667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.102945 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.103217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] PATH_INFO: `/v3/projects/eb10687f73454559a363962e5e309735` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.104416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb10687f73454559a363962e5e309735)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.106312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.151902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb10687f73454559a363962e5e309735', 'request_id': 'req-98fd560c-75b2-4330-9163-eb3c1d94c820'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:29.159198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb10687f73454559a363962e5e309735', 'request_id': 'req-98fd560c-75b2-4330-9163-eb3c1d94c820'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:29.165541 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb10687f73454559a363962e5e309735', 'request_id': 'req-98fd560c-75b2-4330-9163-eb3c1d94c820'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:29.176085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] The token cache is being invalidate because project eb10687f73454559a363962e5e309735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:29.176374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:29.177182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-98fd560c-75b2-4330-9163-eb3c1d94c820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:29.179638 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1236/2492] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:29 2022] DELETE /identity/v3/projects/eb10687f73454559a363962e5e309735 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:29.184334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12644d-53eb-46ad-99d1-9044a5075931 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.184627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12644d-53eb-46ad-99d1-9044a5075931 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.184966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12644d-53eb-46ad-99d1-9044a5075931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.203224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-af12644d-53eb-46ad-99d1-9044a5075931 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:29.204452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-af12644d-53eb-46ad-99d1-9044a5075931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:29.231456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:29.271234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1257/2493] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:36:29.279781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee43e9b-d237-4678-a880-3ae368193663 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.280208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee43e9b-d237-4678-a880-3ae368193663 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.280491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee43e9b-d237-4678-a880-3ae368193663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.301517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-aee43e9b-d237-4678-a880-3ae368193663 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:29.302851 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-aee43e9b-d237-4678-a880-3ae368193663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:29.310817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1237/2494] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:29.315460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12644d-53eb-46ad-99d1-9044a5075931 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.326367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12644d-53eb-46ad-99d1-9044a5075931 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.341812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.342498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.342777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.343053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.343831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:36:29.344136 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.345753 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.358938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1258/2495] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:36:29.363288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee43e9b-d237-4678-a880-3ae368193663 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.375854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee43e9b-d237-4678-a880-3ae368193663 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.381697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20f590-bb1f-41de-8827-4e6165d24181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.382435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20f590-bb1f-41de-8827-4e6165d24181 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.382720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20f590-bb1f-41de-8827-4e6165d24181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.382998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20f590-bb1f-41de-8827-4e6165d24181 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.383726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20f590-bb1f-41de-8827-4e6165d24181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.385648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20f590-bb1f-41de-8827-4e6165d24181 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.425444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1238/2496] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:29 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:36:29.426683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.437171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e421f8-08ac-487d-ac5b-24ea895fe5f4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.494066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd407463-b754-498f-bd3e-8be4217f6448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.494740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd407463-b754-498f-bd3e-8be4217f6448 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.495022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd407463-b754-498f-bd3e-8be4217f6448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.495300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd407463-b754-498f-bd3e-8be4217f6448 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.496029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd407463-b754-498f-bd3e-8be4217f6448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.497953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd407463-b754-498f-bd3e-8be4217f6448 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.518047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1259/2497] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:29 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:29.522179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20f590-bb1f-41de-8827-4e6165d24181 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.532775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20f590-bb1f-41de-8827-4e6165d24181 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.537463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79747fd4-03eb-4217-8781-fe38221e66f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.538126 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79747fd4-03eb-4217-8781-fe38221e66f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.538411 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79747fd4-03eb-4217-8781-fe38221e66f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.538695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79747fd4-03eb-4217-8781-fe38221e66f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.539464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79747fd4-03eb-4217-8781-fe38221e66f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.541342 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79747fd4-03eb-4217-8781-fe38221e66f1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.552236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1239/2498] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:29.556475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd407463-b754-498f-bd3e-8be4217f6448 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.566235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd407463-b754-498f-bd3e-8be4217f6448 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.571736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5dc85-5c26-41f0-9af9-0141595f5366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.572404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5dc85-5c26-41f0-9af9-0141595f5366 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.572692 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5dc85-5c26-41f0-9af9-0141595f5366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.572972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5dc85-5c26-41f0-9af9-0141595f5366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.573711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5dc85-5c26-41f0-9af9-0141595f5366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.575319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5dc85-5c26-41f0-9af9-0141595f5366 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.589262 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1260/2499] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:29.591447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79747fd4-03eb-4217-8781-fe38221e66f1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.601538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79747fd4-03eb-4217-8781-fe38221e66f1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.606029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8196740-95f1-4e1e-8ace-266c6c0fc80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.606899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196740-95f1-4e1e-8ace-266c6c0fc80c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.607177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196740-95f1-4e1e-8ace-266c6c0fc80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.607500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196740-95f1-4e1e-8ace-266c6c0fc80c admin admin] PATH_INFO: `/v3/projects/ea448cad3d9b4528b2db5f82db8f5b8c/users/07df3fae87534ce18a2cdb42548b5d03/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.620521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:29.624080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8196740-95f1-4e1e-8ace-266c6c0fc80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea448cad3d9b4528b2db5f82db8f5b8c, user_id=07df3fae87534ce18a2cdb42548b5d03, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.626888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8196740-95f1-4e1e-8ace-266c6c0fc80c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.641781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1240/2500] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:29 2022] PUT /identity/v3/projects/ea448cad3d9b4528b2db5f82db8f5b8c/users/07df3fae87534ce18a2cdb42548b5d03/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:29.652028 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c8c5-26ba-4495-91e6-1728db528144 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.652321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c8c5-26ba-4495-91e6-1728db528144 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.652601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c8c5-26ba-4495-91e6-1728db528144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.680059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1d34c8c5-26ba-4495-91e6-1728db528144 None None] MFA Rules not processed for user `07df3fae87534ce18a2cdb42548b5d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:29.681261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1d34c8c5-26ba-4495-91e6-1728db528144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:29.745108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1261/2501] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:29.749596 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8196740-95f1-4e1e-8ace-266c6c0fc80c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.809831 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8196740-95f1-4e1e-8ace-266c6c0fc80c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.814469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0d778-b50d-4b2a-84c6-152fd1cb4e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.815143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0d778-b50d-4b2a-84c6-152fd1cb4e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.815456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0d778-b50d-4b2a-84c6-152fd1cb4e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.815734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0d778-b50d-4b2a-84c6-152fd1cb4e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.816419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0d778-b50d-4b2a-84c6-152fd1cb4e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.818835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0d778-b50d-4b2a-84c6-152fd1cb4e14 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.844609 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1241/2502] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:29 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:29.848410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34c8c5-26ba-4495-91e6-1728db528144 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.859018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34c8c5-26ba-4495-91e6-1728db528144 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.876613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:29.915167 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c924a0-ff32-4903-ab23-0bc51c28fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.915921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c924a0-ff32-4903-ab23-0bc51c28fb60 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.916226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c924a0-ff32-4903-ab23-0bc51c28fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.916499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c924a0-ff32-4903-ab23-0bc51c28fb60 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.917184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c924a0-ff32-4903-ab23-0bc51c28fb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.919092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c924a0-ff32-4903-ab23-0bc51c28fb60 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.938749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1262/2503] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:29 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:36:29.943001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0d778-b50d-4b2a-84c6-152fd1cb4e14 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.953138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0d778-b50d-4b2a-84c6-152fd1cb4e14 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.958294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19f009-61cd-482e-94cd-8021e503dd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.959058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f009-61cd-482e-94cd-8021e503dd04 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.959357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f009-61cd-482e-94cd-8021e503dd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.959656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f009-61cd-482e-94cd-8021e503dd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.960387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19f009-61cd-482e-94cd-8021e503dd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.961985 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19f009-61cd-482e-94cd-8021e503dd04 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:29.972299 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1242/2504] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:36:29.976160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c924a0-ff32-4903-ab23-0bc51c28fb60 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:29.985928 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c924a0-ff32-4903-ab23-0bc51c28fb60 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:29.990579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47217120-7016-4ac9-bacd-fe260451640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:29.991231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47217120-7016-4ac9-bacd-fe260451640e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:29.991547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47217120-7016-4ac9-bacd-fe260451640e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:29.991827 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47217120-7016-4ac9-bacd-fe260451640e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:29.992577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47217120-7016-4ac9-bacd-fe260451640e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:29.994168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47217120-7016-4ac9-bacd-fe260451640e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.003789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1263/2505] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:30.008446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19f009-61cd-482e-94cd-8021e503dd04 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.018195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19f009-61cd-482e-94cd-8021e503dd04 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.023485 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea4257-afd6-4a43-bbd1-5a6c92f9b20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.024356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea4257-afd6-4a43-bbd1-5a6c92f9b20d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.024639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea4257-afd6-4a43-bbd1-5a6c92f9b20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.024914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea4257-afd6-4a43-bbd1-5a6c92f9b20d admin admin] PATH_INFO: `/v3/projects/dae6ce3999f84dd7a9e8937ef0c8394b/users/60cdb97e2ba64768a4792f650e1b0ba1/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.040063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea4257-afd6-4a43-bbd1-5a6c92f9b20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae6ce3999f84dd7a9e8937ef0c8394b, user_id=60cdb97e2ba64768a4792f650e1b0ba1, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.042767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea4257-afd6-4a43-bbd1-5a6c92f9b20d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.056364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1243/2506] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:30 2022] PUT /identity/v3/projects/dae6ce3999f84dd7a9e8937ef0c8394b/users/60cdb97e2ba64768a4792f650e1b0ba1/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:30.067228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623d46d-071b-4705-805b-45b79d1bab1f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.067547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623d46d-071b-4705-805b-45b79d1bab1f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.067840 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623d46d-071b-4705-805b-45b79d1bab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.093280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5623d46d-071b-4705-805b-45b79d1bab1f None None] MFA Rules not processed for user `60cdb97e2ba64768a4792f650e1b0ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:30.094472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5623d46d-071b-4705-805b-45b79d1bab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:30.158462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1264/2507] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:30.163647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea4257-afd6-4a43-bbd1-5a6c92f9b20d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.224762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea4257-afd6-4a43-bbd1-5a6c92f9b20d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.229406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3bee1-e18d-4812-a079-0f8308a41bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.230083 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3bee1-e18d-4812-a079-0f8308a41bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.230363 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3bee1-e18d-4812-a079-0f8308a41bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.230657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3bee1-e18d-4812-a079-0f8308a41bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.231409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3bee1-e18d-4812-a079-0f8308a41bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.233300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3bee1-e18d-4812-a079-0f8308a41bc9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.258160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1244/2508] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:30 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:30.262346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623d46d-071b-4705-805b-45b79d1bab1f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.273428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623d46d-071b-4705-805b-45b79d1bab1f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.328568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bfc85-5e25-4632-a703-04b40e4a6ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.329230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bfc85-5e25-4632-a703-04b40e4a6ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.329510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bfc85-5e25-4632-a703-04b40e4a6ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.329782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bfc85-5e25-4632-a703-04b40e4a6ffa admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.330460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bfc85-5e25-4632-a703-04b40e4a6ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.333294 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bfc85-5e25-4632-a703-04b40e4a6ffa admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.353456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1265/2509] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:30 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:30.357462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3bee1-e18d-4812-a079-0f8308a41bc9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.367854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3bee1-e18d-4812-a079-0f8308a41bc9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.372365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858d4a4-2be2-4cda-b0c7-8e27f8bdda6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.373013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d4a4-2be2-4cda-b0c7-8e27f8bdda6d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.373294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d4a4-2be2-4cda-b0c7-8e27f8bdda6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.373570 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d4a4-2be2-4cda-b0c7-8e27f8bdda6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.374288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858d4a4-2be2-4cda-b0c7-8e27f8bdda6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.375933 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858d4a4-2be2-4cda-b0c7-8e27f8bdda6d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.386814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1245/2510] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:30.390802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bfc85-5e25-4632-a703-04b40e4a6ffa admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.401261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bfc85-5e25-4632-a703-04b40e4a6ffa admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.406176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782e4d0-96f1-4c19-9491-f9ea3fb17183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.407047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e4d0-96f1-4c19-9491-f9ea3fb17183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.407321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e4d0-96f1-4c19-9491-f9ea3fb17183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.407616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e4d0-96f1-4c19-9491-f9ea3fb17183 admin admin] PATH_INFO: `/v3/projects/6de5b03682294a218344bd76501880e2/users/d27dddd0a5ec4306baef0ffcc33eac3d/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.422729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782e4d0-96f1-4c19-9491-f9ea3fb17183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de5b03682294a218344bd76501880e2, user_id=d27dddd0a5ec4306baef0ffcc33eac3d, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.425349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782e4d0-96f1-4c19-9491-f9ea3fb17183 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.428904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:30.440305 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1266/2511] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:30 2022] PUT /identity/v3/projects/6de5b03682294a218344bd76501880e2/users/d27dddd0a5ec4306baef0ffcc33eac3d/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:30.444826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858d4a4-2be2-4cda-b0c7-8e27f8bdda6d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.455132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858d4a4-2be2-4cda-b0c7-8e27f8bdda6d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.468381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:30.516891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31922503-d93a-49d6-b496-a60391ee5bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.517573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922503-d93a-49d6-b496-a60391ee5bec admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.517888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922503-d93a-49d6-b496-a60391ee5bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.518173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922503-d93a-49d6-b496-a60391ee5bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.518907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31922503-d93a-49d6-b496-a60391ee5bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.520538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31922503-d93a-49d6-b496-a60391ee5bec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.533513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1246/2512] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:30.535396 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782e4d0-96f1-4c19-9491-f9ea3fb17183 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.546131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782e4d0-96f1-4c19-9491-f9ea3fb17183 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.552336 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.553248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.553533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.553809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] PATH_INFO: `/v3/projects/6de5b03682294a218344bd76501880e2/users/d27dddd0a5ec4306baef0ffcc33eac3d/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.556014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de5b03682294a218344bd76501880e2, user_id=d27dddd0a5ec4306baef0ffcc33eac3d, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.558853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.569900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1267/2513] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:30 2022] PUT /identity/v3/projects/6de5b03682294a218344bd76501880e2/users/d27dddd0a5ec4306baef0ffcc33eac3d/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:30.580481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f69044-e44b-439d-9f74-3389e82c5721 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.580919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f69044-e44b-439d-9f74-3389e82c5721 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.581191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f69044-e44b-439d-9f74-3389e82c5721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.609326 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-43f69044-e44b-439d-9f74-3389e82c5721 None None] MFA Rules not processed for user `d27dddd0a5ec4306baef0ffcc33eac3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:30.610681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-43f69044-e44b-439d-9f74-3389e82c5721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:30.695509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1247/2514] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:30.706677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.710811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:30.713390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f69044-e44b-439d-9f74-3389e82c5721 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:30.714770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-43f69044-e44b-439d-9f74-3389e82c5721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:30.774492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f41fe-f270-4fe2-972e-5621a849c12a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.775519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f69044-e44b-439d-9f74-3389e82c5721 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:30.778431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.779151 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.779280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:30.779562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.779819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.779939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:30.780124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:30.780397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:30.782333 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:30.789677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.791126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.800637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1268/2515] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:30.832470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:30.848649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:30.850193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:30.859992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1248/2516] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:31.304575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:31.306004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:31.317701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9f683-15c2-46ac-ae6f-ee2810695bff service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:31.321334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634733d4-0579-4892-9ec1-9211bbcd21b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:31.321975 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634733d4-0579-4892-9ec1-9211bbcd21b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:31.322255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634733d4-0579-4892-9ec1-9211bbcd21b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:31.322529 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634733d4-0579-4892-9ec1-9211bbcd21b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:31.381757 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634733d4-0579-4892-9ec1-9211bbcd21b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:31.383307 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634733d4-0579-4892-9ec1-9211bbcd21b4 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:31.393434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1269/2517] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:34.732311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:34.733794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:34.818419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447c3ef-2e41-4561-805a-b09be47ac673 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:34.829778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:34.830919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:34.831463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:34.831967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:34.834258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:34.847587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:34.849898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:34.860728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1249/2518] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:35.456120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634733d4-0579-4892-9ec1-9211bbcd21b4 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:35.465734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634733d4-0579-4892-9ec1-9211bbcd21b4 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:35.469344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9017b9-4d6a-485d-8552-1fc1b9525669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:35.469983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9017b9-4d6a-485d-8552-1fc1b9525669 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:35.470270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9017b9-4d6a-485d-8552-1fc1b9525669 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:35.470548 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9017b9-4d6a-485d-8552-1fc1b9525669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:35.530338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9017b9-4d6a-485d-8552-1fc1b9525669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:35.531831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9017b9-4d6a-485d-8552-1fc1b9525669 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:35.541434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1270/2519] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:37.798663 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:37.800157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:37.820591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf7818-d482-46e6-8247-7804f29b35f3 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:37.825534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:37.826706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:37.826991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:37.827283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] PATH_INFO: `/v3/users/10c83f7ad6e94f34b7bd9a335ee52257` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:37.828716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c83f7ad6e94f34b7bd9a335ee52257)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:37.830639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:37.843797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9017b9-4d6a-485d-8552-1fc1b9525669 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:37.859285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9017b9-4d6a-485d-8552-1fc1b9525669 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:37.865932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:37.866817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:37.867103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:37.867409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] PATH_INFO: `/v3/users/07df3fae87534ce18a2cdb42548b5d03` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:37.868782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07df3fae87534ce18a2cdb42548b5d03)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:37.870795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:37.887935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c83f7ad6e94f34b7bd9a335ee52257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.889716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:37.897212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c83f7ad6e94f34b7bd9a335ee52257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.914096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c83f7ad6e94f34b7bd9a335ee52257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.914872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c83f7ad6e94f34b7bd9a335ee52257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.922495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6ae43a-035d-49a4-9f43-ee5e6c937db6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c83f7ad6e94f34b7bd9a335ee52257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.925037 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07df3fae87534ce18a2cdb42548b5d03'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.925674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07df3fae87534ce18a2cdb42548b5d03'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.928853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1250/2520] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:37 2022] DELETE /identity/v3/users/10c83f7ad6e94f34b7bd9a335ee52257 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:37.936632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07df3fae87534ce18a2cdb42548b5d03'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.949951 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07df3fae87534ce18a2cdb42548b5d03'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.950911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07df3fae87534ce18a2cdb42548b5d03'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:37.965943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1271/2521] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:37 2022] DELETE /identity/v3/users/07df3fae87534ce18a2cdb42548b5d03 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:37.970506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0798b9-bfac-4cd7-80b3-64db2d6e2dca None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:37.970871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0798b9-bfac-4cd7-80b3-64db2d6e2dca None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:37.971159 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0798b9-bfac-4cd7-80b3-64db2d6e2dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:37.990315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5e0798b9-bfac-4cd7-80b3-64db2d6e2dca None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:37.991673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5e0798b9-bfac-4cd7-80b3-64db2d6e2dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:38.038433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:38.056932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1251/2522] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:36:38.068352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:38.069843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:38.111525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0798b9-bfac-4cd7-80b3-64db2d6e2dca None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:38.113182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5e0798b9-bfac-4cd7-80b3-64db2d6e2dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:38.126661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0798b9-bfac-4cd7-80b3-64db2d6e2dca None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:38.133827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:38.134821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.135227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.135620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] PATH_INFO: `/v3/projects/c1187f363c0d40ffaebe938d3201e0ad` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.137001 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1187f363c0d40ffaebe938d3201e0ad)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:38.139414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:38.142804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb6629-cdbc-4859-9204-73eb41eaea37 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:38.147119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5206972-7676-4862-962f-a8668fc047f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:38.147992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5206972-7676-4862-962f-a8668fc047f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.148804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5206972-7676-4862-962f-a8668fc047f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.149089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5206972-7676-4862-962f-a8668fc047f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.158456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5206972-7676-4862-962f-a8668fc047f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:38.160033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5206972-7676-4862-962f-a8668fc047f0 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:38.170868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1272/2523] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:38.189291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1187f363c0d40ffaebe938d3201e0ad', 'request_id': 'req-8328e71f-6c04-4a90-9756-92192b545f0b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.198528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1187f363c0d40ffaebe938d3201e0ad', 'request_id': 'req-8328e71f-6c04-4a90-9756-92192b545f0b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.205190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1187f363c0d40ffaebe938d3201e0ad', 'request_id': 'req-8328e71f-6c04-4a90-9756-92192b545f0b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.216915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] The token cache is being invalidate because project c1187f363c0d40ffaebe938d3201e0ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:38.217202 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.217901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.219917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1252/2524] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:38 2022] DELETE /identity/v3/projects/c1187f363c0d40ffaebe938d3201e0ad => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:38.224055 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5206972-7676-4862-962f-a8668fc047f0 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:38.234479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5206972-7676-4862-962f-a8668fc047f0 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:38.297182 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:38.297974 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.298251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.298522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] PATH_INFO: `/v3/users/73dc79b41df1408db9735928da6653df` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.299795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73dc79b41df1408db9735928da6653df)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:38.301890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:38.354728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:38.361882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73dc79b41df1408db9735928da6653df'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.362896 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73dc79b41df1408db9735928da6653df'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.370210 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73dc79b41df1408db9735928da6653df'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.383473 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73dc79b41df1408db9735928da6653df'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.384171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73dc79b41df1408db9735928da6653df'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.398230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1273/2525] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:38 2022] DELETE /identity/v3/users/73dc79b41df1408db9735928da6653df => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:38.451599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:38.452905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:38.466594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328e71f-6c04-4a90-9756-92192b545f0b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:38.529718 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:38.530653 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.530930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.531210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] PATH_INFO: `/v3/projects/ea448cad3d9b4528b2db5f82db8f5b8c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.532428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea448cad3d9b4528b2db5f82db8f5b8c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:38.534630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:38.586308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:38.588372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea448cad3d9b4528b2db5f82db8f5b8c', 'request_id': 'req-368d12b0-8a4c-4480-8f0f-68012ce769e8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.588759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:38.596870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea448cad3d9b4528b2db5f82db8f5b8c', 'request_id': 'req-368d12b0-8a4c-4480-8f0f-68012ce769e8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.603877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea448cad3d9b4528b2db5f82db8f5b8c', 'request_id': 'req-368d12b0-8a4c-4480-8f0f-68012ce769e8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.616402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] The token cache is being invalidate because project ea448cad3d9b4528b2db5f82db8f5b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:38.616758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.617646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.619320 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1253/2526] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:38 2022] DELETE /identity/v3/projects/ea448cad3d9b4528b2db5f82db8f5b8c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:38.622166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f44299-87c4-404a-a5e0-b41e7bd7061d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:38.623112 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:38.624502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:38.630197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:38.684466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:38.685428 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.685703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.685983 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] PATH_INFO: `/v3/projects/619f6038fc644e42bc968395310b90ba` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.687150 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=619f6038fc644e42bc968395310b90ba)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:38.689099 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:38.695792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d12b0-8a4c-4480-8f0f-68012ce769e8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:38.701226 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:38.702091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.702368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.702755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] PATH_INFO: `/v3/users/60cdb97e2ba64768a4792f650e1b0ba1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.703991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cdb97e2ba64768a4792f650e1b0ba1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:38.705886 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:38.745410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '619f6038fc644e42bc968395310b90ba', 'request_id': 'req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.752168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '619f6038fc644e42bc968395310b90ba', 'request_id': 'req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.759738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '619f6038fc644e42bc968395310b90ba', 'request_id': 'req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.760033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cdb97e2ba64768a4792f650e1b0ba1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.769292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cdb97e2ba64768a4792f650e1b0ba1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.772941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] The token cache is being invalidate because project 619f6038fc644e42bc968395310b90ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:38.773250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.773915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d0ce37c7-911d-4c8f-80dd-6a0443ce2a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.775531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1274/2527] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:38 2022] DELETE /identity/v3/projects/619f6038fc644e42bc968395310b90ba => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:38.780706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd188e4-8a2a-4a4d-aa88-6c7c1eb886df None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.781066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd188e4-8a2a-4a4d-aa88-6c7c1eb886df None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.781345 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd188e4-8a2a-4a4d-aa88-6c7c1eb886df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.783024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cdb97e2ba64768a4792f650e1b0ba1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.783878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cdb97e2ba64768a4792f650e1b0ba1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.791290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35621d45-3996-4f79-8635-0d8e31d76476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cdb97e2ba64768a4792f650e1b0ba1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:38.797846 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1254/2528] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:38 2022] DELETE /identity/v3/users/60cdb97e2ba64768a4792f650e1b0ba1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:38.799525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cdd188e4-8a2a-4a4d-aa88-6c7c1eb886df None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:38.800733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cdd188e4-8a2a-4a4d-aa88-6c7c1eb886df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:38.843795 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:38.866437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1275/2529] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:38.874649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687dd8e-bea0-4b69-b7d4-fa371ffd71be None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.875166 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687dd8e-bea0-4b69-b7d4-fa371ffd71be None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.875417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687dd8e-bea0-4b69-b7d4-fa371ffd71be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.895525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e687dd8e-bea0-4b69-b7d4-fa371ffd71be None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:38.896863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e687dd8e-bea0-4b69-b7d4-fa371ffd71be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:38.905442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1255/2530] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:36:38.909524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd188e4-8a2a-4a4d-aa88-6c7c1eb886df None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:38.919519 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd188e4-8a2a-4a4d-aa88-6c7c1eb886df None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:38.924213 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:38.924876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.925172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.925432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.926200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:36:38.926501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:38.928178 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:38.943026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1276/2531] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:36:38.948497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687dd8e-bea0-4b69-b7d4-fa371ffd71be None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:38.966852 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687dd8e-bea0-4b69-b7d4-fa371ffd71be None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:38.971764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:38.972431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:38.972723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:38.973039 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:38.973737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:38.975692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.003380 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1256/2532] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:38 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:39.007140 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:39.017348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8580a7c-eab8-48b0-9e60-462e497f3716 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:39.091571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843a098-11c3-48f2-8f49-f884da2d9d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:39.092269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843a098-11c3-48f2-8f49-f884da2d9d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.092536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843a098-11c3-48f2-8f49-f884da2d9d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.092812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843a098-11c3-48f2-8f49-f884da2d9d24 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.093980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843a098-11c3-48f2-8f49-f884da2d9d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:39.095908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843a098-11c3-48f2-8f49-f884da2d9d24 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.098992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:39.101381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:39.118460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562621c2-f854-4559-8b1d-5ad59614fc70 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:39.122152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1277/2533] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:39 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:39.123656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:39.124532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.124907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.125092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] PATH_INFO: `/v3/projects/dae6ce3999f84dd7a9e8937ef0c8394b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.126566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae6ce3999f84dd7a9e8937ef0c8394b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:39.127237 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843a098-11c3-48f2-8f49-f884da2d9d24 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:39.128524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.141547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843a098-11c3-48f2-8f49-f884da2d9d24 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:39.148452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bcf00-d325-4cd1-9b56-646f5c3c3563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:39.149201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bcf00-d325-4cd1-9b56-646f5c3c3563 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.149319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bcf00-d325-4cd1-9b56-646f5c3c3563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.149595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bcf00-d325-4cd1-9b56-646f5c3c3563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.150541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bcf00-d325-4cd1-9b56-646f5c3c3563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:39.152546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bcf00-d325-4cd1-9b56-646f5c3c3563 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.164148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1278/2534] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:39.168024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bcf00-d325-4cd1-9b56-646f5c3c3563 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:39.177720 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bcf00-d325-4cd1-9b56-646f5c3c3563 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:39.181736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:39.183250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae6ce3999f84dd7a9e8937ef0c8394b', 'request_id': 'req-e53473d8-c5d0-403e-971b-768eb23bd45b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.191510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae6ce3999f84dd7a9e8937ef0c8394b', 'request_id': 'req-e53473d8-c5d0-403e-971b-768eb23bd45b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.199595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae6ce3999f84dd7a9e8937ef0c8394b', 'request_id': 'req-e53473d8-c5d0-403e-971b-768eb23bd45b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.211825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] The token cache is being invalidate because project dae6ce3999f84dd7a9e8937ef0c8394b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:39.212115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.212787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.214354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1257/2535] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:39 2022] DELETE /identity/v3/projects/dae6ce3999f84dd7a9e8937ef0c8394b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:36:39.218116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:39.219884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:39.237988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:39.239316 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.239755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.240035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.240898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:39.242564 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.253153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1279/2536] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:36:39.258077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:39.259769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:39.271736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d28de-5ae8-4fae-b037-eadcd31919a9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:39.276358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e203bf-6c52-4819-b916-51c66fe56991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:39.277228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e203bf-6c52-4819-b916-51c66fe56991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.277510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e203bf-6c52-4819-b916-51c66fe56991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.277789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e203bf-6c52-4819-b916-51c66fe56991 admin admin] PATH_INFO: `/v3/projects/3785f78e6d1048cfa0b5b123fdd909c8/users/4a827bedc4b84508bb7770a09efee3f5/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.285710 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53473d8-c5d0-403e-971b-768eb23bd45b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:39.290820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:39.291747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.292029 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.292302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] PATH_INFO: `/v3/users/d27dddd0a5ec4306baef0ffcc33eac3d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.292925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e203bf-6c52-4819-b916-51c66fe56991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3785f78e6d1048cfa0b5b123fdd909c8, user_id=4a827bedc4b84508bb7770a09efee3f5, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:39.293566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27dddd0a5ec4306baef0ffcc33eac3d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:39.295593 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.295705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e203bf-6c52-4819-b916-51c66fe56991 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.309478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1280/2537] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:39 2022] PUT /identity/v3/projects/3785f78e6d1048cfa0b5b123fdd909c8/users/4a827bedc4b84508bb7770a09efee3f5/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:36:39.314296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdee846-da39-496c-8ef8-a94cea2aee53 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.314589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdee846-da39-496c-8ef8-a94cea2aee53 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.314873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdee846-da39-496c-8ef8-a94cea2aee53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.333113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-afdee846-da39-496c-8ef8-a94cea2aee53 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:39.334318 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-afdee846-da39-496c-8ef8-a94cea2aee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:39.345961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27dddd0a5ec4306baef0ffcc33eac3d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.354112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:39.354334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27dddd0a5ec4306baef0ffcc33eac3d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.367756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27dddd0a5ec4306baef0ffcc33eac3d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.368431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27dddd0a5ec4306baef0ffcc33eac3d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.375520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27dddd0a5ec4306baef0ffcc33eac3d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.382378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1258/2538] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:39 2022] DELETE /identity/v3/users/d27dddd0a5ec4306baef0ffcc33eac3d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:36:39.397411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1281/2539] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:36:39.408591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:39.410308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:39.475331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0a5d-1b7b-4726-a292-f4d1227a2d3c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:39.479082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499b963-86ea-4654-8a4a-25076008e185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:39.479810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499b963-86ea-4654-8a4a-25076008e185 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.480735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499b963-86ea-4654-8a4a-25076008e185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.481049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499b963-86ea-4654-8a4a-25076008e185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.491290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:39.549022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499b963-86ea-4654-8a4a-25076008e185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:39.550704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499b963-86ea-4654-8a4a-25076008e185 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.561428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1259/2540] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:39.759260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdee846-da39-496c-8ef8-a94cea2aee53 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:39.760741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-afdee846-da39-496c-8ef8-a94cea2aee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:39.774410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdee846-da39-496c-8ef8-a94cea2aee53 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:39.782612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:39.783292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.783628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.783902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] PATH_INFO: `/v3/projects/6de5b03682294a218344bd76501880e2` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.785146 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de5b03682294a218344bd76501880e2)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:39.787820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:39.860822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de5b03682294a218344bd76501880e2', 'request_id': 'req-67741947-0877-4365-a630-2134f36cbce6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.867777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de5b03682294a218344bd76501880e2', 'request_id': 'req-67741947-0877-4365-a630-2134f36cbce6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.873539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de5b03682294a218344bd76501880e2', 'request_id': 'req-67741947-0877-4365-a630-2134f36cbce6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.886463 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] The token cache is being invalidate because project 6de5b03682294a218344bd76501880e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:39.886776 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.887449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67741947-0877-4365-a630-2134f36cbce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:39.889143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1282/2541] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:39 2022] DELETE /identity/v3/projects/6de5b03682294a218344bd76501880e2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:39.895008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef1ba-49d1-4d79-ba37-76227a2cf8cd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.895906 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef1ba-49d1-4d79-ba37-76227a2cf8cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.896258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef1ba-49d1-4d79-ba37-76227a2cf8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:39.917817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cd9ef1ba-49d1-4d79-ba37-76227a2cf8cd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:39.918746 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cd9ef1ba-49d1-4d79-ba37-76227a2cf8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:39.990258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1260/2542] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:36:39.999409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e0e3-5d4c-44ef-9b42-345556479dcd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:39.999708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e0e3-5d4c-44ef-9b42-345556479dcd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:39.999999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e0e3-5d4c-44ef-9b42-345556479dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.018987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ac28e0e3-5d4c-44ef-9b42-345556479dcd None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:40.021450 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ac28e0e3-5d4c-44ef-9b42-345556479dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:40.033872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1283/2543] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:40.035826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ef1ba-49d1-4d79-ba37-76227a2cf8cd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:40.049066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ef1ba-49d1-4d79-ba37-76227a2cf8cd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:40.055139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:40.055810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.056161 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.056740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.057306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:36:40.057656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:40.059406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:40.075447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1261/2544] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:36:40.076680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28e0e3-5d4c-44ef-9b42-345556479dcd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:40.088008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28e0e3-5d4c-44ef-9b42-345556479dcd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:40.092881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172999c8-9aba-4567-ad8c-c0dffaf52327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:40.093741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172999c8-9aba-4567-ad8c-c0dffaf52327 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.094089 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172999c8-9aba-4567-ad8c-c0dffaf52327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.094427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172999c8-9aba-4567-ad8c-c0dffaf52327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.095220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172999c8-9aba-4567-ad8c-c0dffaf52327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:40.097705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172999c8-9aba-4567-ad8c-c0dffaf52327 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:40.130876 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1284/2545] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:40 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:40.134742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:40.144580 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655cdd4-24e4-4884-ba48-ffae6086b648 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:40.214606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea75d5-a093-4c10-b949-9f953f785df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:40.215511 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea75d5-a093-4c10-b949-9f953f785df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.215799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea75d5-a093-4c10-b949-9f953f785df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.216076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea75d5-a093-4c10-b949-9f953f785df7 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.216776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea75d5-a093-4c10-b949-9f953f785df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:40.218698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea75d5-a093-4c10-b949-9f953f785df7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:40.240656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1262/2546] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:40 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:40.245235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172999c8-9aba-4567-ad8c-c0dffaf52327 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:40.258703 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172999c8-9aba-4567-ad8c-c0dffaf52327 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:40.264257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fafa8-7026-4eae-853b-bc55906a832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:40.265033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fafa8-7026-4eae-853b-bc55906a832f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.265416 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fafa8-7026-4eae-853b-bc55906a832f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.265767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fafa8-7026-4eae-853b-bc55906a832f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.266679 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fafa8-7026-4eae-853b-bc55906a832f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:40.268967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fafa8-7026-4eae-853b-bc55906a832f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:40.291052 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1285/2547] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:36:40.292542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea75d5-a093-4c10-b949-9f953f785df7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:40.304740 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea75d5-a093-4c10-b949-9f953f785df7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:40.309926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea88a-97a7-4041-9320-4f4205605b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:40.310613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea88a-97a7-4041-9320-4f4205605b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.310898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea88a-97a7-4041-9320-4f4205605b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.311177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea88a-97a7-4041-9320-4f4205605b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.311982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ea88a-97a7-4041-9320-4f4205605b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:40.313585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ea88a-97a7-4041-9320-4f4205605b75 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:40.327923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1263/2548] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:40.333030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fafa8-7026-4eae-853b-bc55906a832f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:40.353599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fafa8-7026-4eae-853b-bc55906a832f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:40.360650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:40.361954 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.362346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.362676 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] PATH_INFO: `/v3/projects/df34ce51b82e4727a40b614b1f926845/users/13aa68466b3d40a2bd871e330d466bf9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.389157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df34ce51b82e4727a40b614b1f926845, user_id=13aa68466b3d40a2bd871e330d466bf9, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:40.393405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:40.414445 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1286/2549] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:40 2022] PUT /identity/v3/projects/df34ce51b82e4727a40b614b1f926845/users/13aa68466b3d40a2bd871e330d466bf9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:36:40.431387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41818af7-c346-4cd1-83e4-4b8abea4d4fa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.431779 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41818af7-c346-4cd1-83e4-4b8abea4d4fa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.432812 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41818af7-c346-4cd1-83e4-4b8abea4d4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.460187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:40.471984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-41818af7-c346-4cd1-83e4-4b8abea4d4fa None None] MFA Rules not processed for user `13aa68466b3d40a2bd871e330d466bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:40.474261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-41818af7-c346-4cd1-83e4-4b8abea4d4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:40.592691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1264/2550] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:36:40.609963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:40.612152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:40.672736 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:40.691043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c321e-3ab5-4d6b-8630-8a8a0bf43887 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:40.696652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971ffc5-4739-4cf1-bced-5cc2acf64d19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:40.698469 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971ffc5-4739-4cf1-bced-5cc2acf64d19 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.698941 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971ffc5-4739-4cf1-bced-5cc2acf64d19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.699431 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971ffc5-4739-4cf1-bced-5cc2acf64d19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.719990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971ffc5-4739-4cf1-bced-5cc2acf64d19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:40.723961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971ffc5-4739-4cf1-bced-5cc2acf64d19 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:40.736658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1287/2551] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:36:40.762783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41818af7-c346-4cd1-83e4-4b8abea4d4fa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:40.850364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41818af7-c346-4cd1-83e4-4b8abea4d4fa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:40.856036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:40.857503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:40.857977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:40.858476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] PATH_INFO: `/v3/users/13aa68466b3d40a2bd871e330d466bf9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:40.860301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=13aa68466b3d40a2bd871e330d466bf9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:40.863944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:40.943417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13aa68466b3d40a2bd871e330d466bf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:40.954093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13aa68466b3d40a2bd871e330d466bf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:40.971765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13aa68466b3d40a2bd871e330d466bf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:40.972488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13aa68466b3d40a2bd871e330d466bf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:40.981450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13aa68466b3d40a2bd871e330d466bf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:40.995442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1265/2552] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:40 2022] DELETE /identity/v3/users/13aa68466b3d40a2bd871e330d466bf9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:41.000274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be27f8-1c79-404f-a834-c134841b1a98 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:41.000778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be27f8-1c79-404f-a834-c134841b1a98 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:41.001047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be27f8-1c79-404f-a834-c134841b1a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:41.028085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-74be27f8-1c79-404f-a834-c134841b1a98 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:41.029384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-74be27f8-1c79-404f-a834-c134841b1a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:41.133484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1288/2553] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:41.152503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:41.154194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:41.181190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:41.225122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63219af4-d6f3-455f-b0ec-590e24ee92dc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:41.228869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15383b37-9ec1-4e82-91e0-623de4048519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:41.229809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15383b37-9ec1-4e82-91e0-623de4048519 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:41.230091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15383b37-9ec1-4e82-91e0-623de4048519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:41.230373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15383b37-9ec1-4e82-91e0-623de4048519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:41.241744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15383b37-9ec1-4e82-91e0-623de4048519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:41.243081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15383b37-9ec1-4e82-91e0-623de4048519 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:41.256430 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1266/2554] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:36:42.009166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be27f8-1c79-404f-a834-c134841b1a98 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:42.010638 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-74be27f8-1c79-404f-a834-c134841b1a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:42.043122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be27f8-1c79-404f-a834-c134841b1a98 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:42.049228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:42.050126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.050407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.051323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] PATH_INFO: `/v3/projects/df34ce51b82e4727a40b614b1f926845` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.052695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=df34ce51b82e4727a40b614b1f926845)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:42.056295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:42.118300 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df34ce51b82e4727a40b614b1f926845', 'request_id': 'req-1f768373-1615-4c2b-8b11-f3b184d3e39b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:42.125002 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df34ce51b82e4727a40b614b1f926845', 'request_id': 'req-1f768373-1615-4c2b-8b11-f3b184d3e39b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:42.132218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df34ce51b82e4727a40b614b1f926845', 'request_id': 'req-1f768373-1615-4c2b-8b11-f3b184d3e39b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:42.149195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] The token cache is being invalidate because project df34ce51b82e4727a40b614b1f926845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:42.149553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:42.150661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1f768373-1615-4c2b-8b11-f3b184d3e39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:42.152270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1289/2555] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:42 2022] DELETE /identity/v3/projects/df34ce51b82e4727a40b614b1f926845 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:42.157384 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485fc7-8bb1-4e66-931a-3fa1187195df None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.157948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485fc7-8bb1-4e66-931a-3fa1187195df None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.158058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485fc7-8bb1-4e66-931a-3fa1187195df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.182076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1b485fc7-8bb1-4e66-931a-3fa1187195df None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:42.183826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1b485fc7-8bb1-4e66-931a-3fa1187195df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:42.257839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1267/2556] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:42.265523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c639c-e968-43a8-9d55-a0dd7ea00760 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.265820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c639c-e968-43a8-9d55-a0dd7ea00760 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.266143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c639c-e968-43a8-9d55-a0dd7ea00760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.295837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-fd6c639c-e968-43a8-9d55-a0dd7ea00760 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:42.297478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fd6c639c-e968-43a8-9d55-a0dd7ea00760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:42.309808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1290/2557] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:42.313973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485fc7-8bb1-4e66-931a-3fa1187195df None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:42.324915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485fc7-8bb1-4e66-931a-3fa1187195df None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:42.330804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:42.331769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.332649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.332938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.333804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:36:42.334187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:42.335917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:42.354939 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1268/2558] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:36:42.356065 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c639c-e968-43a8-9d55-a0dd7ea00760 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:42.366412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c639c-e968-43a8-9d55-a0dd7ea00760 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:42.371601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3e94a-1d92-4238-b985-d1b1ba706230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:42.372352 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3e94a-1d92-4238-b985-d1b1ba706230 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.372715 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3e94a-1d92-4238-b985-d1b1ba706230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.373069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3e94a-1d92-4238-b985-d1b1ba706230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.373818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3e94a-1d92-4238-b985-d1b1ba706230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:42.376275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3e94a-1d92-4238-b985-d1b1ba706230 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:42.417171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1291/2559] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:42 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:36:42.421696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:42.434567 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030386c1-3a17-46d0-b6ef-5c6bbdcb6161 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:42.524487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790d10-fae5-402e-a8b7-b9225f2a34eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:42.526451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790d10-fae5-402e-a8b7-b9225f2a34eb admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.526954 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790d10-fae5-402e-a8b7-b9225f2a34eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.528244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790d10-fae5-402e-a8b7-b9225f2a34eb admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.529547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be790d10-fae5-402e-a8b7-b9225f2a34eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:42.532834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be790d10-fae5-402e-a8b7-b9225f2a34eb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:42.565403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1269/2560] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:42 2022] POST /identity/v3/users => generated 285 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:36:42.569279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3e94a-1d92-4238-b985-d1b1ba706230 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:42.584571 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3e94a-1d92-4238-b985-d1b1ba706230 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:42.593701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1f4bd-47a1-4107-8c78-ebcffb76c3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:42.595080 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1f4bd-47a1-4107-8c78-ebcffb76c3da admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.595474 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1f4bd-47a1-4107-8c78-ebcffb76c3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.595850 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1f4bd-47a1-4107-8c78-ebcffb76c3da admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.597517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1f4bd-47a1-4107-8c78-ebcffb76c3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:42.600696 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1f4bd-47a1-4107-8c78-ebcffb76c3da admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:42.617082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1292/2561] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:42.621089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790d10-fae5-402e-a8b7-b9225f2a34eb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:42.632137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790d10-fae5-402e-a8b7-b9225f2a34eb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:42.636909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d6cc6-924c-4ee3-ae0c-debbbf17e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:42.637705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6cc6-924c-4ee3-ae0c-debbbf17e7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.638068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6cc6-924c-4ee3-ae0c-debbbf17e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.638350 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6cc6-924c-4ee3-ae0c-debbbf17e7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.639220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d6cc6-924c-4ee3-ae0c-debbbf17e7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:42.640965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d6cc6-924c-4ee3-ae0c-debbbf17e7ca admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:42.659518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1270/2562] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:36:42.664270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1f4bd-47a1-4107-8c78-ebcffb76c3da admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:42.678299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1f4bd-47a1-4107-8c78-ebcffb76c3da admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:42.684595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:42.685989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.685989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.686187 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] PATH_INFO: `/v3/projects/16275a7776744f2a9ddd4016e626c201/users/0afd07ccf7904dc0978b774cfc877c74/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.709510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16275a7776744f2a9ddd4016e626c201, user_id=0afd07ccf7904dc0978b774cfc877c74, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:42.723365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:42.741824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1293/2563] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:42 2022] PUT /identity/v3/projects/16275a7776744f2a9ddd4016e626c201/users/0afd07ccf7904dc0978b774cfc877c74/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:42.755036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10333a7c-8c48-4364-b305-0117a9b0f81b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.755681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10333a7c-8c48-4364-b305-0117a9b0f81b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.756080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10333a7c-8c48-4364-b305-0117a9b0f81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:42.791368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-10333a7c-8c48-4364-b305-0117a9b0f81b None None] MFA Rules not processed for user `0afd07ccf7904dc0978b774cfc877c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:42.792764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-10333a7c-8c48-4364-b305-0117a9b0f81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:42.815396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:42.881959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1271/2564] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:42.895797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:42.897608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:42.909157 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:42.994545 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f295d9-9c43-4083-9d59-5006ed2fb012 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:42.998343 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b8dd344e8e43ec9cec222737f98ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:42.999004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:42.999289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:42.999599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:43.010215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:43.011629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:43.022057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1294/2565] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:42 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:44.187685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8147d4e-cfdc-4560-961c-b5d7611f33cd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:44.187685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8147d4e-cfdc-4560-961c-b5d7611f33cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:44.187685 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8147d4e-cfdc-4560-961c-b5d7611f33cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:44.220937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d8147d4e-cfdc-4560-961c-b5d7611f33cd None None] MFA Rules not processed for user `4a827bedc4b84508bb7770a09efee3f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:44.222467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d8147d4e-cfdc-4560-961c-b5d7611f33cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:44.301356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1272/2566] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:44.306020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:44.307491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:44.393977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404772f8-373a-49fc-86eb-c33d26ba45b9 service glance] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:44.399507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecefce-8e69-42d8-aee8-baf385d8e218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:44.400204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecefce-8e69-42d8-aee8-baf385d8e218 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:44.400525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecefce-8e69-42d8-aee8-baf385d8e218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:44.400872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecefce-8e69-42d8-aee8-baf385d8e218 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:44.401597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecefce-8e69-42d8-aee8-baf385d8e218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:44.403612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecefce-8e69-42d8-aee8-baf385d8e218 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:44.429581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1295/2567] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:44 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:44.433591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8147d4e-cfdc-4560-961c-b5d7611f33cd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:44.446050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8147d4e-cfdc-4560-961c-b5d7611f33cd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:44.518268 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48cc09-7460-4031-8fae-bcbee8448ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:44.518957 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48cc09-7460-4031-8fae-bcbee8448ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:44.519265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48cc09-7460-4031-8fae-bcbee8448ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:44.519784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48cc09-7460-4031-8fae-bcbee8448ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:44.520509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48cc09-7460-4031-8fae-bcbee8448ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:44.522469 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48cc09-7460-4031-8fae-bcbee8448ec3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:44.547420 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1273/2568] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:44 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:36:44.548581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecefce-8e69-42d8-aee8-baf385d8e218 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:44.561259 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecefce-8e69-42d8-aee8-baf385d8e218 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:44.568242 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e310a-d267-4603-aca9-65df4aa1b8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:44.569200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e310a-d267-4603-aca9-65df4aa1b8fb admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:44.569690 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e310a-d267-4603-aca9-65df4aa1b8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:44.570122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e310a-d267-4603-aca9-65df4aa1b8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:44.571250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e310a-d267-4603-aca9-65df4aa1b8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:44.573869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e310a-d267-4603-aca9-65df4aa1b8fb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:44.591587 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1296/2569] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:44.596657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48cc09-7460-4031-8fae-bcbee8448ec3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:44.608037 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48cc09-7460-4031-8fae-bcbee8448ec3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:44.612807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76b3a9-c88b-4977-bfa1-22ad7b6fa07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:44.614573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76b3a9-c88b-4977-bfa1-22ad7b6fa07e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:44.614855 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76b3a9-c88b-4977-bfa1-22ad7b6fa07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:44.615131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76b3a9-c88b-4977-bfa1-22ad7b6fa07e admin admin] PATH_INFO: `/v3/projects/a52ea793d7ce46f59ed1392b1b987de5/users/9f9a2922508546ac9fbbcf643e5340c0/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:44.644990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76b3a9-c88b-4977-bfa1-22ad7b6fa07e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52ea793d7ce46f59ed1392b1b987de5, user_id=9f9a2922508546ac9fbbcf643e5340c0, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:44.662138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76b3a9-c88b-4977-bfa1-22ad7b6fa07e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:44.677323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1274/2570] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:44 2022] PUT /identity/v3/projects/a52ea793d7ce46f59ed1392b1b987de5/users/9f9a2922508546ac9fbbcf643e5340c0/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:44.681429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e310a-d267-4603-aca9-65df4aa1b8fb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:44.702189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e310a-d267-4603-aca9-65df4aa1b8fb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:44.765484 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:44.766228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:44.766517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:44.766800 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:44.767584 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:44.769266 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:44.781595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1297/2571] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:44.786698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76b3a9-c88b-4977-bfa1-22ad7b6fa07e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:44.797536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76b3a9-c88b-4977-bfa1-22ad7b6fa07e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:44.803608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd624849-e6d0-4b96-a00c-e763dc3efa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:44.805093 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd624849-e6d0-4b96-a00c-e763dc3efa1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:44.805618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd624849-e6d0-4b96-a00c-e763dc3efa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:44.805915 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd624849-e6d0-4b96-a00c-e763dc3efa1c admin admin] PATH_INFO: `/v3/projects/a52ea793d7ce46f59ed1392b1b987de5/users/9f9a2922508546ac9fbbcf643e5340c0/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:44.809672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd624849-e6d0-4b96-a00c-e763dc3efa1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52ea793d7ce46f59ed1392b1b987de5, user_id=9f9a2922508546ac9fbbcf643e5340c0, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:44.813478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd624849-e6d0-4b96-a00c-e763dc3efa1c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:44.824415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:44.832624 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1275/2572] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:44 2022] PUT /identity/v3/projects/a52ea793d7ce46f59ed1392b1b987de5/users/9f9a2922508546ac9fbbcf643e5340c0/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:49.150206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:49.152270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:49.241496 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:49.388837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460955c-d06f-4df0-b2a0-62f287e190f4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:49.547619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:49.548220 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:49.548506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:49.548783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:49.551867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:49.581779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:49.583734 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:49.607097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1298/2573] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:50.455197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b7c2-95e6-436c-9c22-d319d10d23cb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:50.455676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b7c2-95e6-436c-9c22-d319d10d23cb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:50.455860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b7c2-95e6-436c-9c22-d319d10d23cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:50.488555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ff28b7c2-95e6-436c-9c22-d319d10d23cb None None] MFA Rules not processed for user `9f9a2922508546ac9fbbcf643e5340c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:50.490248 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ff28b7c2-95e6-436c-9c22-d319d10d23cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:50.594858 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1276/2574] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:50.610954 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:50.623936 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb352-7f2e-4f9b-863d-d4ef58b7de38 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:50.632329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:50.633232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:50.633675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:50.634107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:50.708526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:50.710470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:50.724522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1299/2575] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:54.681522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28b7c2-95e6-436c-9c22-d319d10d23cb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:54.683659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ff28b7c2-95e6-436c-9c22-d319d10d23cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:54.765191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28b7c2-95e6-436c-9c22-d319d10d23cb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:54.768902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:54.770117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:54.770952 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:54.771146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:54.773056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:54.860813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:54.862716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:54.877754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1277/2576] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:36:58.182129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:58.183651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:58.197818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7768341-9f15-4ef2-b9bd-92a54b0b98bb service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:58.211272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:58.212284 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:58.212781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:58.213047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] PATH_INFO: `/v3/users/710b406ba65646ed94c3f9d356b161f1` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:58.214349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=710b406ba65646ed94c3f9d356b161f1)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:58.216383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:58.260575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:58.270248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710b406ba65646ed94c3f9d356b161f1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.270952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710b406ba65646ed94c3f9d356b161f1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.278412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710b406ba65646ed94c3f9d356b161f1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.291299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710b406ba65646ed94c3f9d356b161f1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.292062 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a0c84710-452b-4daa-aa6e-e0df3d7ba4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710b406ba65646ed94c3f9d356b161f1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.304952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1300/2577] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:58 2022] DELETE /identity/v3/users/710b406ba65646ed94c3f9d356b161f1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:36:58.481682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:58.495038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131151a8-d651-4290-85bf-09bf403257e8 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:58.534443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:58.558734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:58.559946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:58.560225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:58.560497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] PATH_INFO: `/v3/projects/afaaf8bde7134efc80c77d707bb309bc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:58.561698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afaaf8bde7134efc80c77d707bb309bc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:58.563672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:58.617041 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afaaf8bde7134efc80c77d707bb309bc', 'request_id': 'req-c42e0f07-a52a-4a71-b9c6-b742b962c013'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.624111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afaaf8bde7134efc80c77d707bb309bc', 'request_id': 'req-c42e0f07-a52a-4a71-b9c6-b742b962c013'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.632225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afaaf8bde7134efc80c77d707bb309bc', 'request_id': 'req-c42e0f07-a52a-4a71-b9c6-b742b962c013'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.642651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] The token cache is being invalidate because project afaaf8bde7134efc80c77d707bb309bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:36:58.642934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.643631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c42e0f07-a52a-4a71-b9c6-b742b962c013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:36:58.646019 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1278/2578] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:36:58 2022] DELETE /identity/v3/projects/afaaf8bde7134efc80c77d707bb309bc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:58.650933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c6140-23d1-4ea0-b441-a14bb0006747 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:58.651227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c6140-23d1-4ea0-b441-a14bb0006747 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:58.651589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c6140-23d1-4ea0-b441-a14bb0006747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:58.669686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1f7c6140-23d1-4ea0-b441-a14bb0006747 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:58.670883 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1f7c6140-23d1-4ea0-b441-a14bb0006747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:58.738853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1301/2579] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:58.746879 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d748197-ca68-44c5-a77d-74a207f8d358 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:58.747085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d748197-ca68-44c5-a77d-74a207f8d358 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:58.747382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d748197-ca68-44c5-a77d-74a207f8d358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:58.766070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7d748197-ca68-44c5-a77d-74a207f8d358 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:58.767261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7d748197-ca68-44c5-a77d-74a207f8d358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:58.775231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1279/2580] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:58.779576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c6140-23d1-4ea0-b441-a14bb0006747 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:58.789580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c6140-23d1-4ea0-b441-a14bb0006747 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:58.794047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:58.794705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:58.794978 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:58.795247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:58.795997 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:36:58.796302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:58.797901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:58.809786 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1302/2581] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:36:58.814371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d748197-ca68-44c5-a77d-74a207f8d358 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:58.824546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d748197-ca68-44c5-a77d-74a207f8d358 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:58.829228 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2789e-7b20-40ff-8461-ee19ab016233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:58.829885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2789e-7b20-40ff-8461-ee19ab016233 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:58.830167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2789e-7b20-40ff-8461-ee19ab016233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:58.830443 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2789e-7b20-40ff-8461-ee19ab016233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:58.831151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2789e-7b20-40ff-8461-ee19ab016233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:58.833308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2789e-7b20-40ff-8461-ee19ab016233 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:58.859127 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1280/2582] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:58 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:58.864369 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:58.874465 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39c28-4d6c-4abc-b9e3-59345f49e855 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:58.934521 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f92ef-a02e-410b-a68e-9c7fd2cbcb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:58.935197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f92ef-a02e-410b-a68e-9c7fd2cbcb1a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:58.935514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f92ef-a02e-410b-a68e-9c7fd2cbcb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:58.935790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f92ef-a02e-410b-a68e-9c7fd2cbcb1a admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:58.936478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f92ef-a02e-410b-a68e-9c7fd2cbcb1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:58.938566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f92ef-a02e-410b-a68e-9c7fd2cbcb1a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:58.959785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1303/2583] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:58 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:58.963891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2789e-7b20-40ff-8461-ee19ab016233 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:58.974003 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2789e-7b20-40ff-8461-ee19ab016233 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:58.978798 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e1ae6-3e93-4ba2-8d13-a6e9de389ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:58.979461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e1ae6-3e93-4ba2-8d13-a6e9de389ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:58.979749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e1ae6-3e93-4ba2-8d13-a6e9de389ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:58.980026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e1ae6-3e93-4ba2-8d13-a6e9de389ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:58.980770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e1ae6-3e93-4ba2-8d13-a6e9de389ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:58.982428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e1ae6-3e93-4ba2-8d13-a6e9de389ca2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:58.993890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1281/2584] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:58.997958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f92ef-a02e-410b-a68e-9c7fd2cbcb1a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.009689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f92ef-a02e-410b-a68e-9c7fd2cbcb1a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.014534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28b47f-1b16-443a-8252-d53253f125a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.015269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b47f-1b16-443a-8252-d53253f125a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.015600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b47f-1b16-443a-8252-d53253f125a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.015877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b47f-1b16-443a-8252-d53253f125a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.016614 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28b47f-1b16-443a-8252-d53253f125a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.018227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28b47f-1b16-443a-8252-d53253f125a4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.028842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1304/2585] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:59.032808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e1ae6-3e93-4ba2-8d13-a6e9de389ca2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.043538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e1ae6-3e93-4ba2-8d13-a6e9de389ca2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.048265 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca0351-4f7d-468f-94cc-afac224c7809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.049163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca0351-4f7d-468f-94cc-afac224c7809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.049442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca0351-4f7d-468f-94cc-afac224c7809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.049715 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca0351-4f7d-468f-94cc-afac224c7809 admin admin] PATH_INFO: `/v3/projects/24a791adfdcf47af8ce8489937a894c2/users/1dfa67779c4e40a2a03f11b19f939bad/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.073422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca0351-4f7d-468f-94cc-afac224c7809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a791adfdcf47af8ce8489937a894c2, user_id=1dfa67779c4e40a2a03f11b19f939bad, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.076180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca0351-4f7d-468f-94cc-afac224c7809 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.091478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1282/2586] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:59 2022] PUT /identity/v3/projects/24a791adfdcf47af8ce8489937a894c2/users/1dfa67779c4e40a2a03f11b19f939bad/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:59.102288 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee700d7d-ecf1-4317-83e3-1592a95a73b0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.102652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee700d7d-ecf1-4317-83e3-1592a95a73b0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.102995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee700d7d-ecf1-4317-83e3-1592a95a73b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.131476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ee700d7d-ecf1-4317-83e3-1592a95a73b0 None None] MFA Rules not processed for user `1dfa67779c4e40a2a03f11b19f939bad`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:59.132681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ee700d7d-ecf1-4317-83e3-1592a95a73b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:59.140741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:59.199159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1305/2587] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:59.203651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca0351-4f7d-468f-94cc-afac224c7809 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.275797 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca0351-4f7d-468f-94cc-afac224c7809 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.280547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f3c0a-a858-4a4c-af0f-bffb3b37cd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.281219 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3c0a-a858-4a4c-af0f-bffb3b37cd9d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.281506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3c0a-a858-4a4c-af0f-bffb3b37cd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.281840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3c0a-a858-4a4c-af0f-bffb3b37cd9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.282478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f3c0a-a858-4a4c-af0f-bffb3b37cd9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.285699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f3c0a-a858-4a4c-af0f-bffb3b37cd9d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.289977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:36:59.319395 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1283/2588] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:36:59 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:59.321893 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee700d7d-ecf1-4317-83e3-1592a95a73b0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.333289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee700d7d-ecf1-4317-83e3-1592a95a73b0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.392926 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459a112-160e-4ef4-945e-6c128cc7a130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.394148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459a112-160e-4ef4-945e-6c128cc7a130 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.394435 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459a112-160e-4ef4-945e-6c128cc7a130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.394706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459a112-160e-4ef4-945e-6c128cc7a130 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.395551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459a112-160e-4ef4-945e-6c128cc7a130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.397860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459a112-160e-4ef4-945e-6c128cc7a130 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.433326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1306/2589] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:36:59 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:36:59.437880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f3c0a-a858-4a4c-af0f-bffb3b37cd9d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.451172 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f3c0a-a858-4a4c-af0f-bffb3b37cd9d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.455871 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9df37-2e7f-4592-a10d-a9e0871e48f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.457922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9df37-2e7f-4592-a10d-a9e0871e48f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.458197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9df37-2e7f-4592-a10d-a9e0871e48f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.458472 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9df37-2e7f-4592-a10d-a9e0871e48f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.459210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9df37-2e7f-4592-a10d-a9e0871e48f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.460845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9df37-2e7f-4592-a10d-a9e0871e48f9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.474720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1284/2590] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:36:59.478759 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459a112-160e-4ef4-945e-6c128cc7a130 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.490832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459a112-160e-4ef4-945e-6c128cc7a130 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.495524 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5a2f6-a221-4c56-9bd5-ec4600619282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.496408 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5a2f6-a221-4c56-9bd5-ec4600619282 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.496688 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5a2f6-a221-4c56-9bd5-ec4600619282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.496971 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5a2f6-a221-4c56-9bd5-ec4600619282 admin admin] PATH_INFO: `/v3/projects/7f9f5a8bc1e6496c9f4d3ae80d3c47e7/users/33a44de4239746738cb24ba012fb234a/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.512520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5a2f6-a221-4c56-9bd5-ec4600619282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9f5a8bc1e6496c9f4d3ae80d3c47e7, user_id=33a44de4239746738cb24ba012fb234a, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.515137 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5a2f6-a221-4c56-9bd5-ec4600619282 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.528942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1307/2591] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:59 2022] PUT /identity/v3/projects/7f9f5a8bc1e6496c9f4d3ae80d3c47e7/users/33a44de4239746738cb24ba012fb234a/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:59.533087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9df37-2e7f-4592-a10d-a9e0871e48f9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.544077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9df37-2e7f-4592-a10d-a9e0871e48f9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.601050 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0458ba-a1cb-4c2a-b679-ee657a0cf433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.601736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0458ba-a1cb-4c2a-b679-ee657a0cf433 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.602022 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0458ba-a1cb-4c2a-b679-ee657a0cf433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.602324 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0458ba-a1cb-4c2a-b679-ee657a0cf433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.603057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0458ba-a1cb-4c2a-b679-ee657a0cf433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.604695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0458ba-a1cb-4c2a-b679-ee657a0cf433 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.615907 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1285/2592] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:36:59.619784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5a2f6-a221-4c56-9bd5-ec4600619282 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.632699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5a2f6-a221-4c56-9bd5-ec4600619282 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.637456 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.638344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.638618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.638885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] PATH_INFO: `/v3/projects/7f9f5a8bc1e6496c9f4d3ae80d3c47e7/users/33a44de4239746738cb24ba012fb234a/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.640892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9f5a8bc1e6496c9f4d3ae80d3c47e7, user_id=33a44de4239746738cb24ba012fb234a, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.643575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.658329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1308/2593] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:36:59 2022] PUT /identity/v3/projects/7f9f5a8bc1e6496c9f4d3ae80d3c47e7/users/33a44de4239746738cb24ba012fb234a/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:36:59.669755 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d2047-d9f9-4fd0-a272-b34c2598682e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.670048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d2047-d9f9-4fd0-a272-b34c2598682e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.670329 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d2047-d9f9-4fd0-a272-b34c2598682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.697735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4f3d2047-d9f9-4fd0-a272-b34c2598682e None None] MFA Rules not processed for user `33a44de4239746738cb24ba012fb234a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:36:59.698944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4f3d2047-d9f9-4fd0-a272-b34c2598682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:59.769687 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1286/2594] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:36:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:36:59.780516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:36:59.782236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:36:59.840185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedfac6-9ecc-48da-9846-c7029892f13c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:36:59.844380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14844712-2571-45c1-810e-c2728914e934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:36:59.845024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844712-2571-45c1-810e-c2728914e934 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:36:59.845302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844712-2571-45c1-810e-c2728914e934 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:36:59.845575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844712-2571-45c1-810e-c2728914e934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:36:59.855549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14844712-2571-45c1-810e-c2728914e934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:36:59.856979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14844712-2571-45c1-810e-c2728914e934 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:36:59.868350 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1309/2595] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:36:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:36:59.999464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d2047-d9f9-4fd0-a272-b34c2598682e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:00.059382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:00.064140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d2047-d9f9-4fd0-a272-b34c2598682e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:00.069960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:00.070835 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:00.071119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:00.071445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] PATH_INFO: `/v3/users/1dfa67779c4e40a2a03f11b19f939bad` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:00.072630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dfa67779c4e40a2a03f11b19f939bad)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:00.074551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:00.120744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dfa67779c4e40a2a03f11b19f939bad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.129619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dfa67779c4e40a2a03f11b19f939bad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.142552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dfa67779c4e40a2a03f11b19f939bad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.143229 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dfa67779c4e40a2a03f11b19f939bad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.151018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dfa67779c4e40a2a03f11b19f939bad'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.156910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1287/2596] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:36:59 2022] DELETE /identity/v3/users/1dfa67779c4e40a2a03f11b19f939bad => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:00.161666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aca343-64d0-4d5d-bcde-c09d5326212b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:00.161955 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aca343-64d0-4d5d-bcde-c09d5326212b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:00.162231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aca343-64d0-4d5d-bcde-c09d5326212b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:00.168781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:00.181193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-85aca343-64d0-4d5d-bcde-c09d5326212b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:00.182403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-85aca343-64d0-4d5d-bcde-c09d5326212b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:00.245050 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1310/2597] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:00.255716 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:00.257255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:00.314084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488d36a-7054-420e-a8f1-0096a40bc8ba admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:00.317985 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12fd5a-f7c8-4b6c-b8c5-56a3d7e3ac77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:00.318639 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12fd5a-f7c8-4b6c-b8c5-56a3d7e3ac77 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:00.319043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12fd5a-f7c8-4b6c-b8c5-56a3d7e3ac77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:00.319211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12fd5a-f7c8-4b6c-b8c5-56a3d7e3ac77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:00.330286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12fd5a-f7c8-4b6c-b8c5-56a3d7e3ac77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:00.332482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12fd5a-f7c8-4b6c-b8c5-56a3d7e3ac77 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:00.342315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1288/2598] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:00.620937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aca343-64d0-4d5d-bcde-c09d5326212b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:00.622391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-85aca343-64d0-4d5d-bcde-c09d5326212b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:00.640442 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aca343-64d0-4d5d-bcde-c09d5326212b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:00.648763 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:00.650296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:00.650784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:00.651285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] PATH_INFO: `/v3/projects/24a791adfdcf47af8ce8489937a894c2` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:00.653415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a791adfdcf47af8ce8489937a894c2)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:00.656166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:00.724740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a791adfdcf47af8ce8489937a894c2', 'request_id': 'req-7e8a621b-5302-44a1-bf76-ceb554603089'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.731209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a791adfdcf47af8ce8489937a894c2', 'request_id': 'req-7e8a621b-5302-44a1-bf76-ceb554603089'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.736716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a791adfdcf47af8ce8489937a894c2', 'request_id': 'req-7e8a621b-5302-44a1-bf76-ceb554603089'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.749665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] The token cache is being invalidate because project 24a791adfdcf47af8ce8489937a894c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:00.749950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.750558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.752191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1311/2599] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:00 2022] DELETE /identity/v3/projects/24a791adfdcf47af8ce8489937a894c2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:00.757043 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12fd5a-f7c8-4b6c-b8c5-56a3d7e3ac77 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:00.766489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12fd5a-f7c8-4b6c-b8c5-56a3d7e3ac77 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:00.824332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:00.825217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:00.825500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:00.825774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] PATH_INFO: `/v3/users/33a44de4239746738cb24ba012fb234a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:00.826962 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a44de4239746738cb24ba012fb234a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:00.828911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:00.851932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:00.882206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a44de4239746738cb24ba012fb234a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.888664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a44de4239746738cb24ba012fb234a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.906081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a44de4239746738cb24ba012fb234a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.906899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a44de4239746738cb24ba012fb234a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.914160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dc82ee7b-e5e3-4241-9405-4d2a92554e75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a44de4239746738cb24ba012fb234a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:00.920094 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1289/2600] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:00 2022] DELETE /identity/v3/users/33a44de4239746738cb24ba012fb234a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:01.197141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:01.198902 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:01.213102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a621b-5302-44a1-bf76-ceb554603089 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:01.276290 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:01.277170 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.277444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.277713 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] PATH_INFO: `/v3/projects/7f9f5a8bc1e6496c9f4d3ae80d3c47e7` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.278923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9f5a8bc1e6496c9f4d3ae80d3c47e7)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:01.281635 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:01.318132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:01.332478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9f5a8bc1e6496c9f4d3ae80d3c47e7', 'request_id': 'req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:01.338718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9f5a8bc1e6496c9f4d3ae80d3c47e7', 'request_id': 'req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:01.344122 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9f5a8bc1e6496c9f4d3ae80d3c47e7', 'request_id': 'req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:01.358394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] The token cache is being invalidate because project 7f9f5a8bc1e6496c9f4d3ae80d3c47e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:01.358677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:01.359325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7563d9bb-312a-4eb8-b8d7-39e5c802ecb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:01.360889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1312/2601] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:01 2022] DELETE /identity/v3/projects/7f9f5a8bc1e6496c9f4d3ae80d3c47e7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:01.366590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c633c-81f2-420d-9666-29aab43a36da None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.366884 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c633c-81f2-420d-9666-29aab43a36da None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.367165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c633c-81f2-420d-9666-29aab43a36da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.385615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-303c633c-81f2-420d-9666-29aab43a36da None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:01.387133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-303c633c-81f2-420d-9666-29aab43a36da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:01.449380 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1290/2602] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:01.457616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05560fb7-be09-4534-b31a-157b4bab4914 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.457898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05560fb7-be09-4534-b31a-157b4bab4914 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.458172 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05560fb7-be09-4534-b31a-157b4bab4914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.477808 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-05560fb7-be09-4534-b31a-157b4bab4914 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:01.479197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-05560fb7-be09-4534-b31a-157b4bab4914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:01.489137 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1313/2603] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:01.493633 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c633c-81f2-420d-9666-29aab43a36da None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:01.505554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c633c-81f2-420d-9666-29aab43a36da None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:01.510090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:01.510741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.511040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.511304 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.512076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:37:01.512385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:01.513991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:01.526292 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1291/2604] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 17:37:01.530684 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05560fb7-be09-4534-b31a-157b4bab4914 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:01.540597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05560fb7-be09-4534-b31a-157b4bab4914 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:01.545310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079df80-8d78-4f7e-8761-3a6623b86ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:01.545971 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079df80-8d78-4f7e-8761-3a6623b86ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.546276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079df80-8d78-4f7e-8761-3a6623b86ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.546597 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079df80-8d78-4f7e-8761-3a6623b86ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.547329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079df80-8d78-4f7e-8761-3a6623b86ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:01.549358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079df80-8d78-4f7e-8761-3a6623b86ed0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:01.576717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1314/2605] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:01 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:01.581109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:01.591536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fd2ff-665a-420d-99ee-a1d6328c33bb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:01.647481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6ddd-ebc2-4623-b97b-207ea5f21c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:01.648069 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6ddd-ebc2-4623-b97b-207ea5f21c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.648354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6ddd-ebc2-4623-b97b-207ea5f21c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.648632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6ddd-ebc2-4623-b97b-207ea5f21c4d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.649335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6ddd-ebc2-4623-b97b-207ea5f21c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:01.651255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6ddd-ebc2-4623-b97b-207ea5f21c4d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:01.671169 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1292/2606] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:01 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:01.676143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079df80-8d78-4f7e-8761-3a6623b86ed0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:01.687252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079df80-8d78-4f7e-8761-3a6623b86ed0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:01.695073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93421a4-9b67-4be6-8f7e-cbec9758fcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:01.695785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93421a4-9b67-4be6-8f7e-cbec9758fcce admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.696068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93421a4-9b67-4be6-8f7e-cbec9758fcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.696344 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93421a4-9b67-4be6-8f7e-cbec9758fcce admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.697110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93421a4-9b67-4be6-8f7e-cbec9758fcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:01.698758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93421a4-9b67-4be6-8f7e-cbec9758fcce admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:01.710247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1315/2607] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:37:01.714446 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6ddd-ebc2-4623-b97b-207ea5f21c4d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:01.725606 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6ddd-ebc2-4623-b97b-207ea5f21c4d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:01.730244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c775e45-03a3-4e00-b658-da330896e80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:01.730909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c775e45-03a3-4e00-b658-da330896e80b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.732119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c775e45-03a3-4e00-b658-da330896e80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.732119 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c775e45-03a3-4e00-b658-da330896e80b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.732881 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c775e45-03a3-4e00-b658-da330896e80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:01.734476 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c775e45-03a3-4e00-b658-da330896e80b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:01.745083 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1293/2608] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:01.749654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93421a4-9b67-4be6-8f7e-cbec9758fcce admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:01.761448 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93421a4-9b67-4be6-8f7e-cbec9758fcce admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:01.769612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863844f2-0ce2-46b4-ba55-861b19f7ac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:01.769612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863844f2-0ce2-46b4-ba55-861b19f7ac66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.769612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863844f2-0ce2-46b4-ba55-861b19f7ac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.769612 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863844f2-0ce2-46b4-ba55-861b19f7ac66 admin admin] PATH_INFO: `/v3/projects/0eaf89a2893b482da80e858ccc15a997/users/6b27726263aa46339a077b7f670ca85e/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.783480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863844f2-0ce2-46b4-ba55-861b19f7ac66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eaf89a2893b482da80e858ccc15a997, user_id=6b27726263aa46339a077b7f670ca85e, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:01.786848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863844f2-0ce2-46b4-ba55-861b19f7ac66 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:01.800822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1316/2609] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:01 2022] PUT /identity/v3/projects/0eaf89a2893b482da80e858ccc15a997/users/6b27726263aa46339a077b7f670ca85e/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:37:01.811750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d4b22-9e5b-4250-ae6b-f04a6ecd58a3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.812317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d4b22-9e5b-4250-ae6b-f04a6ecd58a3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.812317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d4b22-9e5b-4250-ae6b-f04a6ecd58a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.819245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:01.844102 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0f4d4b22-9e5b-4250-ae6b-f04a6ecd58a3 None None] MFA Rules not processed for user `6b27726263aa46339a077b7f670ca85e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:01.845301 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0f4d4b22-9e5b-4250-ae6b-f04a6ecd58a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:01.911856 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1294/2610] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:01.916407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863844f2-0ce2-46b4-ba55-861b19f7ac66 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:01.979814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863844f2-0ce2-46b4-ba55-861b19f7ac66 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:01.984818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4438d12-d621-4e64-a799-1fa44138051f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:01.985543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4438d12-d621-4e64-a799-1fa44138051f admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:01.985820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4438d12-d621-4e64-a799-1fa44138051f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:01.986096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4438d12-d621-4e64-a799-1fa44138051f admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:01.986847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4438d12-d621-4e64-a799-1fa44138051f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:01.989051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4438d12-d621-4e64-a799-1fa44138051f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:02.014214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1317/2611] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:01 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:02.018250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d4b22-9e5b-4250-ae6b-f04a6ecd58a3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:02.029538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d4b22-9e5b-4250-ae6b-f04a6ecd58a3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:02.091830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11633bd2-1646-483a-b447-6bf37f04861b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:02.092665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11633bd2-1646-483a-b447-6bf37f04861b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:02.092969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11633bd2-1646-483a-b447-6bf37f04861b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:02.093241 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11633bd2-1646-483a-b447-6bf37f04861b admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:02.093929 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11633bd2-1646-483a-b447-6bf37f04861b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:02.095864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11633bd2-1646-483a-b447-6bf37f04861b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:02.116982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1295/2612] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:02 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:02.121125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4438d12-d621-4e64-a799-1fa44138051f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:02.132646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4438d12-d621-4e64-a799-1fa44138051f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:02.137320 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64b040-1aa8-41fa-af42-9dec73ec17be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:02.137982 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64b040-1aa8-41fa-af42-9dec73ec17be admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:02.138264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64b040-1aa8-41fa-af42-9dec73ec17be admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:02.138537 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64b040-1aa8-41fa-af42-9dec73ec17be admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:02.139261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64b040-1aa8-41fa-af42-9dec73ec17be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:02.140892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64b040-1aa8-41fa-af42-9dec73ec17be admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:02.152267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1318/2613] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:02.156277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11633bd2-1646-483a-b447-6bf37f04861b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:02.166312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11633bd2-1646-483a-b447-6bf37f04861b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:02.171162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541335e-d284-43c5-9c49-a072b56a494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:02.172066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541335e-d284-43c5-9c49-a072b56a494b admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:02.172349 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541335e-d284-43c5-9c49-a072b56a494b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:02.172627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541335e-d284-43c5-9c49-a072b56a494b admin admin] PATH_INFO: `/v3/projects/c4a45356096c4e2aa20505ba5f286ae2/users/fac048347f7d46d39bda9f8ef8c3422f/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:02.189497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541335e-d284-43c5-9c49-a072b56a494b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a45356096c4e2aa20505ba5f286ae2, user_id=fac048347f7d46d39bda9f8ef8c3422f, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:02.193338 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541335e-d284-43c5-9c49-a072b56a494b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:02.207691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1296/2614] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:02 2022] PUT /identity/v3/projects/c4a45356096c4e2aa20505ba5f286ae2/users/fac048347f7d46d39bda9f8ef8c3422f/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:02.211813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64b040-1aa8-41fa-af42-9dec73ec17be admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:02.221629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64b040-1aa8-41fa-af42-9dec73ec17be admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:02.248930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:02.284209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd4200-dc09-46ff-bd2d-491493096af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:02.284987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd4200-dc09-46ff-bd2d-491493096af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:02.285274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd4200-dc09-46ff-bd2d-491493096af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:02.285551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd4200-dc09-46ff-bd2d-491493096af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:02.286272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd4200-dc09-46ff-bd2d-491493096af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:02.287901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd4200-dc09-46ff-bd2d-491493096af9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:02.297516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1319/2615] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:02.301946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541335e-d284-43c5-9c49-a072b56a494b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:02.312109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541335e-d284-43c5-9c49-a072b56a494b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:02.317539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:02.318453 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:02.318739 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:02.319021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] PATH_INFO: `/v3/projects/c4a45356096c4e2aa20505ba5f286ae2/users/fac048347f7d46d39bda9f8ef8c3422f/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:02.321121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a45356096c4e2aa20505ba5f286ae2, user_id=fac048347f7d46d39bda9f8ef8c3422f, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:02.323799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:02.335033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1297/2616] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:02 2022] PUT /identity/v3/projects/c4a45356096c4e2aa20505ba5f286ae2/users/fac048347f7d46d39bda9f8ef8c3422f/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:02.346166 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8132f3-6dbd-4ecc-bedc-234fbdc6798a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:02.346455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8132f3-6dbd-4ecc-bedc-234fbdc6798a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:02.347368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8132f3-6dbd-4ecc-bedc-234fbdc6798a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:02.373787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9f8132f3-6dbd-4ecc-bedc-234fbdc6798a None None] MFA Rules not processed for user `fac048347f7d46d39bda9f8ef8c3422f`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:02.374996 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9f8132f3-6dbd-4ecc-bedc-234fbdc6798a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:02.449782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1320/2617] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:02.463610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:02.465308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:02.521542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:02.527861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a31322-aa6b-4830-83ab-e00f085a509b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:02.531977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c51df-1594-44df-a0f7-ccc9f7e7af95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:02.532655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c51df-1594-44df-a0f7-ccc9f7e7af95 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:02.532937 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c51df-1594-44df-a0f7-ccc9f7e7af95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:02.533208 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c51df-1594-44df-a0f7-ccc9f7e7af95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:02.543258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c51df-1594-44df-a0f7-ccc9f7e7af95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:02.544706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c51df-1594-44df-a0f7-ccc9f7e7af95 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:02.553935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1298/2618] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:03.116523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8132f3-6dbd-4ecc-bedc-234fbdc6798a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:03.201299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8132f3-6dbd-4ecc-bedc-234fbdc6798a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:03.206411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:03.207675 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:03.207950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:03.208221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] PATH_INFO: `/v3/users/6b27726263aa46339a077b7f670ca85e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:03.209407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b27726263aa46339a077b7f670ca85e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:03.211327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:03.258887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b27726263aa46339a077b7f670ca85e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.259565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b27726263aa46339a077b7f670ca85e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.263948 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:03.267472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b27726263aa46339a077b7f670ca85e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.280125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b27726263aa46339a077b7f670ca85e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.280784 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b27726263aa46339a077b7f670ca85e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.296085 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1321/2619] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:03 2022] DELETE /identity/v3/users/6b27726263aa46339a077b7f670ca85e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:37:03.300714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30fe42-7a6a-44a7-9dac-80c1deacbbd0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:03.301143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30fe42-7a6a-44a7-9dac-80c1deacbbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:03.301255 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30fe42-7a6a-44a7-9dac-80c1deacbbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:03.318327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6a30fe42-7a6a-44a7-9dac-80c1deacbbd0 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:03.319607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6a30fe42-7a6a-44a7-9dac-80c1deacbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:03.380153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1299/2620] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:03.389874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:03.391270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:03.449120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeff743-3448-43ea-b74e-2d74e19acabf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:03.452672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec145d-9c0b-48ca-b4ed-26ce7b0568cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:03.453312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec145d-9c0b-48ca-b4ed-26ce7b0568cd service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:03.453592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec145d-9c0b-48ca-b4ed-26ce7b0568cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:03.453868 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec145d-9c0b-48ca-b4ed-26ce7b0568cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:03.471168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec145d-9c0b-48ca-b4ed-26ce7b0568cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:03.473835 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec145d-9c0b-48ca-b4ed-26ce7b0568cd service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:03.485271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1322/2621] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:03.766559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30fe42-7a6a-44a7-9dac-80c1deacbbd0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:03.768002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6a30fe42-7a6a-44a7-9dac-80c1deacbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:03.782300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30fe42-7a6a-44a7-9dac-80c1deacbbd0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:03.788084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:03.789090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:03.789402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:03.789869 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] PATH_INFO: `/v3/projects/0eaf89a2893b482da80e858ccc15a997` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:03.791118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eaf89a2893b482da80e858ccc15a997)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:03.793185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:03.848134 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eaf89a2893b482da80e858ccc15a997', 'request_id': 'req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.856143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eaf89a2893b482da80e858ccc15a997', 'request_id': 'req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.862679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eaf89a2893b482da80e858ccc15a997', 'request_id': 'req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.873465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] The token cache is being invalidate because project 0eaf89a2893b482da80e858ccc15a997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:03.873830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.874619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.876274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1300/2622] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:03 2022] DELETE /identity/v3/projects/0eaf89a2893b482da80e858ccc15a997 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:03.881656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec145d-9c0b-48ca-b4ed-26ce7b0568cd service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:03.891215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec145d-9c0b-48ca-b4ed-26ce7b0568cd service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:03.946577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:03.947480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:03.947760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:03.948030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] PATH_INFO: `/v3/users/fac048347f7d46d39bda9f8ef8c3422f` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:03.949204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac048347f7d46d39bda9f8ef8c3422f)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:03.951111 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:03.997074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac048347f7d46d39bda9f8ef8c3422f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:03.997656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac048347f7d46d39bda9f8ef8c3422f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.005586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac048347f7d46d39bda9f8ef8c3422f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.017367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac048347f7d46d39bda9f8ef8c3422f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.018009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ad45057b-50f9-474b-933c-b030ae046810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac048347f7d46d39bda9f8ef8c3422f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.019738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:04.032257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1323/2623] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:03 2022] DELETE /identity/v3/users/fac048347f7d46d39bda9f8ef8c3422f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:04.317140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:04.318652 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:04.332625 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4043d-eedd-4ec1-add6-b2e9eff48dcb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:04.390396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:04.391263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.391582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.391861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] PATH_INFO: `/v3/projects/c4a45356096c4e2aa20505ba5f286ae2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.393045 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a45356096c4e2aa20505ba5f286ae2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:04.395063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:04.411676 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:04.442592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a45356096c4e2aa20505ba5f286ae2', 'request_id': 'req-1e398286-e75a-4adc-9797-f072c42e94dc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.450206 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a45356096c4e2aa20505ba5f286ae2', 'request_id': 'req-1e398286-e75a-4adc-9797-f072c42e94dc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.456519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a45356096c4e2aa20505ba5f286ae2', 'request_id': 'req-1e398286-e75a-4adc-9797-f072c42e94dc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.466673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] The token cache is being invalidate because project c4a45356096c4e2aa20505ba5f286ae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:04.466961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.467615 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e398286-e75a-4adc-9797-f072c42e94dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:04.469156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1301/2624] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:04 2022] DELETE /identity/v3/projects/c4a45356096c4e2aa20505ba5f286ae2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:04.474770 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d7c64-54ca-469d-932c-bcd6db9867e7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.475253 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d7c64-54ca-469d-932c-bcd6db9867e7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.475381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d7c64-54ca-469d-932c-bcd6db9867e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.493767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b32d7c64-54ca-469d-932c-bcd6db9867e7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:04.495051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b32d7c64-54ca-469d-932c-bcd6db9867e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:04.556193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1324/2625] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:04.563870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acdaa6-7743-462d-a6e7-e08687fdccda None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.564168 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acdaa6-7743-462d-a6e7-e08687fdccda None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.564454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acdaa6-7743-462d-a6e7-e08687fdccda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.582534 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-23acdaa6-7743-462d-a6e7-e08687fdccda None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:04.583872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-23acdaa6-7743-462d-a6e7-e08687fdccda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:04.594015 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1302/2626] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:04.598908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d7c64-54ca-469d-932c-bcd6db9867e7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:04.610020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d7c64-54ca-469d-932c-bcd6db9867e7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:04.615580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:04.616267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.616546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.616818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.617568 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:37:04.617872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:04.619506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:04.631273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1325/2627] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:37:04.635597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acdaa6-7743-462d-a6e7-e08687fdccda None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:04.645006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acdaa6-7743-462d-a6e7-e08687fdccda None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:04.649756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2355a-8e4d-4f6c-95f6-a47c7a7cbf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:04.650557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2355a-8e4d-4f6c-95f6-a47c7a7cbf3f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.650708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2355a-8e4d-4f6c-95f6-a47c7a7cbf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.650988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2355a-8e4d-4f6c-95f6-a47c7a7cbf3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.651744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2355a-8e4d-4f6c-95f6-a47c7a7cbf3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:04.653775 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2355a-8e4d-4f6c-95f6-a47c7a7cbf3f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:04.683628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1303/2628] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:04 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:37:04.686372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:04.696813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9244e2-20a0-4207-b192-33f309d7cfbd admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:04.757534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a497e0-84c4-4f1a-9058-322daff7b349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:04.758205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a497e0-84c4-4f1a-9058-322daff7b349 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.758486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a497e0-84c4-4f1a-9058-322daff7b349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.758761 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a497e0-84c4-4f1a-9058-322daff7b349 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.759497 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a497e0-84c4-4f1a-9058-322daff7b349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:04.761412 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a497e0-84c4-4f1a-9058-322daff7b349 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:04.783231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1326/2629] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:04 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:04.787948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2355a-8e4d-4f6c-95f6-a47c7a7cbf3f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:04.798608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2355a-8e4d-4f6c-95f6-a47c7a7cbf3f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:04.803179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539ea2-a72d-4527-8571-422e9a1bc021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:04.803876 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539ea2-a72d-4527-8571-422e9a1bc021 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.804162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539ea2-a72d-4527-8571-422e9a1bc021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.804439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539ea2-a72d-4527-8571-422e9a1bc021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.805198 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539ea2-a72d-4527-8571-422e9a1bc021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:04.806789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539ea2-a72d-4527-8571-422e9a1bc021 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:04.817360 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1304/2630] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:04.821230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a497e0-84c4-4f1a-9058-322daff7b349 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:04.831700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a497e0-84c4-4f1a-9058-322daff7b349 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:04.836475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89124c4f-e7b7-45b3-ac16-ab6a9790685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:04.837128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124c4f-e7b7-45b3-ac16-ab6a9790685e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.837410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124c4f-e7b7-45b3-ac16-ab6a9790685e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.837689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124c4f-e7b7-45b3-ac16-ab6a9790685e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.838414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89124c4f-e7b7-45b3-ac16-ab6a9790685e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:04.840042 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89124c4f-e7b7-45b3-ac16-ab6a9790685e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:04.851627 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1327/2631] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:04.855674 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539ea2-a72d-4527-8571-422e9a1bc021 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:04.865795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539ea2-a72d-4527-8571-422e9a1bc021 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:04.870451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e1c42-44b4-4519-93e1-ec75096f5f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:04.871328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e1c42-44b4-4519-93e1-ec75096f5f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.871636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e1c42-44b4-4519-93e1-ec75096f5f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.871910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e1c42-44b4-4519-93e1-ec75096f5f22 admin admin] PATH_INFO: `/v3/projects/fdc656383fed4892b7b50b2bb1a68354/users/a251cbb8ab734970a5913fdf536077d9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.887910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e1c42-44b4-4519-93e1-ec75096f5f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc656383fed4892b7b50b2bb1a68354, user_id=a251cbb8ab734970a5913fdf536077d9, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:04.891326 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e1c42-44b4-4519-93e1-ec75096f5f22 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:04.906923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1305/2632] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:04 2022] PUT /identity/v3/projects/fdc656383fed4892b7b50b2bb1a68354/users/a251cbb8ab734970a5913fdf536077d9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:04.918060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ab5b1-18fe-43d7-853e-c49922095bb0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:04.918373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ab5b1-18fe-43d7-853e-c49922095bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:04.918792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ab5b1-18fe-43d7-853e-c49922095bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:04.951550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-114ab5b1-18fe-43d7-853e-c49922095bb0 None None] MFA Rules not processed for user `a251cbb8ab734970a5913fdf536077d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:04.952756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-114ab5b1-18fe-43d7-853e-c49922095bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:04.996631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:05.017775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1328/2633] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:05.022777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e1c42-44b4-4519-93e1-ec75096f5f22 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:05.087692 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e1c42-44b4-4519-93e1-ec75096f5f22 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:05.092552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d739bd-6b2e-49b1-8267-75cef5c3552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:05.093244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d739bd-6b2e-49b1-8267-75cef5c3552b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.093528 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d739bd-6b2e-49b1-8267-75cef5c3552b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.093802 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d739bd-6b2e-49b1-8267-75cef5c3552b admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.094496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d739bd-6b2e-49b1-8267-75cef5c3552b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:05.096444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d739bd-6b2e-49b1-8267-75cef5c3552b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:05.120771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1306/2634] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:05 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:05.124825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ab5b1-18fe-43d7-853e-c49922095bb0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:05.136599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ab5b1-18fe-43d7-853e-c49922095bb0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:05.204477 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d92b1-c837-4013-b1c6-970e2ca20720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:05.205153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d92b1-c837-4013-b1c6-970e2ca20720 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.205437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d92b1-c837-4013-b1c6-970e2ca20720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.205712 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d92b1-c837-4013-b1c6-970e2ca20720 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.207038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d92b1-c837-4013-b1c6-970e2ca20720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:05.209036 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d92b1-c837-4013-b1c6-970e2ca20720 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:05.228724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1329/2635] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:05 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:05.232607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d739bd-6b2e-49b1-8267-75cef5c3552b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:05.242935 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d739bd-6b2e-49b1-8267-75cef5c3552b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:05.247759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece52a7e-790a-4c65-939b-559c5c1cb873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:05.248474 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece52a7e-790a-4c65-939b-559c5c1cb873 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.248747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece52a7e-790a-4c65-939b-559c5c1cb873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.249046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece52a7e-790a-4c65-939b-559c5c1cb873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.249768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece52a7e-790a-4c65-939b-559c5c1cb873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:05.251380 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece52a7e-790a-4c65-939b-559c5c1cb873 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:05.262870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1307/2636] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:05.266965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d92b1-c837-4013-b1c6-970e2ca20720 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:05.277376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d92b1-c837-4013-b1c6-970e2ca20720 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:05.281960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3442867-d790-4a71-af63-39e1260513e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:05.282831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442867-d790-4a71-af63-39e1260513e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.283110 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442867-d790-4a71-af63-39e1260513e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.283418 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442867-d790-4a71-af63-39e1260513e2 admin admin] PATH_INFO: `/v3/projects/16d47512828447798aa866017c3aa83b/users/ad663679ba4140e1a618c9bce151b889/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.299189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3442867-d790-4a71-af63-39e1260513e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d47512828447798aa866017c3aa83b, user_id=ad663679ba4140e1a618c9bce151b889, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:05.301865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3442867-d790-4a71-af63-39e1260513e2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:05.313985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1330/2637] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:05 2022] PUT /identity/v3/projects/16d47512828447798aa866017c3aa83b/users/ad663679ba4140e1a618c9bce151b889/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:05.318498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece52a7e-790a-4c65-939b-559c5c1cb873 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:05.329503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece52a7e-790a-4c65-939b-559c5c1cb873 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:05.339153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:05.388698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85ed66-f768-4ace-bff6-4ce1bb44b834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:05.389389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ed66-f768-4ace-bff6-4ce1bb44b834 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.389669 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ed66-f768-4ace-bff6-4ce1bb44b834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.389942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ed66-f768-4ace-bff6-4ce1bb44b834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.390659 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85ed66-f768-4ace-bff6-4ce1bb44b834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:05.392262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85ed66-f768-4ace-bff6-4ce1bb44b834 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:05.402808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1308/2638] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:05.406989 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3442867-d790-4a71-af63-39e1260513e2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:05.417372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3442867-d790-4a71-af63-39e1260513e2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:05.421979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:05.422860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.423145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.423464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] PATH_INFO: `/v3/projects/16d47512828447798aa866017c3aa83b/users/ad663679ba4140e1a618c9bce151b889/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.425495 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d47512828447798aa866017c3aa83b, user_id=ad663679ba4140e1a618c9bce151b889, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:05.428192 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:05.438296 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1331/2639] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:05 2022] PUT /identity/v3/projects/16d47512828447798aa866017c3aa83b/users/ad663679ba4140e1a618c9bce151b889/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:37:05.452899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa56d4-55e5-4ddc-a1a6-b4e6fc87e6de None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.453187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa56d4-55e5-4ddc-a1a6-b4e6fc87e6de None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.453464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa56d4-55e5-4ddc-a1a6-b4e6fc87e6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.481390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-12aa56d4-55e5-4ddc-a1a6-b4e6fc87e6de None None] MFA Rules not processed for user `ad663679ba4140e1a618c9bce151b889`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:05.482582 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-12aa56d4-55e5-4ddc-a1a6-b4e6fc87e6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:05.550509 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1309/2640] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:05.561680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:05.563499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:05.625393 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83934ae-3966-4f22-b7d2-9bc6cd2ecc29 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:05.630005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9c65e-d0d2-453e-ac57-e2c1d8010f78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:05.630642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9c65e-d0d2-453e-ac57-e2c1d8010f78 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.630918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9c65e-d0d2-453e-ac57-e2c1d8010f78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.631191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9c65e-d0d2-453e-ac57-e2c1d8010f78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.640071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9c65e-d0d2-453e-ac57-e2c1d8010f78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:05.641480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9c65e-d0d2-453e-ac57-e2c1d8010f78 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:05.650686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1332/2641] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:05.737955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa56d4-55e5-4ddc-a1a6-b4e6fc87e6de None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:05.747660 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa56d4-55e5-4ddc-a1a6-b4e6fc87e6de None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:05.751243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea740b-c783-4792-ad29-b47c0aa6eaf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:05.751916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea740b-c783-4792-ad29-b47c0aa6eaf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:05.752273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea740b-c783-4792-ad29-b47c0aa6eaf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:05.752551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea740b-c783-4792-ad29-b47c0aa6eaf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:05.809190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea740b-c783-4792-ad29-b47c0aa6eaf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:05.810617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea740b-c783-4792-ad29-b47c0aa6eaf6 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:05.820298 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1310/2642] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:06.075183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9c65e-d0d2-453e-ac57-e2c1d8010f78 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:06.131910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:06.152654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9c65e-d0d2-453e-ac57-e2c1d8010f78 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:06.157291 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:06.162388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:06.162388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:06.162388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] PATH_INFO: `/v3/users/a251cbb8ab734970a5913fdf536077d9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:06.162388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a251cbb8ab734970a5913fdf536077d9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:06.162388 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:06.208605 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a251cbb8ab734970a5913fdf536077d9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.209255 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a251cbb8ab734970a5913fdf536077d9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.216059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a251cbb8ab734970a5913fdf536077d9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.227779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a251cbb8ab734970a5913fdf536077d9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.228417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a251cbb8ab734970a5913fdf536077d9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.242464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1333/2643] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:06 2022] DELETE /identity/v3/users/a251cbb8ab734970a5913fdf536077d9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:06.247085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d05bce-6590-4c01-9c8f-32ca310a6ab7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:06.247416 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d05bce-6590-4c01-9c8f-32ca310a6ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:06.247725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d05bce-6590-4c01-9c8f-32ca310a6ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:06.265898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-42d05bce-6590-4c01-9c8f-32ca310a6ab7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:06.267087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42d05bce-6590-4c01-9c8f-32ca310a6ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:06.294949 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:06.325642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1311/2644] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:06.335207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:06.336674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:06.393097 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896ace-7aa7-4dbb-887a-110cfc14efdb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:06.397206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42b966-e35a-46d2-9218-fe97c781c0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:06.397843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42b966-e35a-46d2-9218-fe97c781c0dc service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:06.398128 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42b966-e35a-46d2-9218-fe97c781c0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:06.398401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42b966-e35a-46d2-9218-fe97c781c0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:06.407633 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42b966-e35a-46d2-9218-fe97c781c0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:06.409630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42b966-e35a-46d2-9218-fe97c781c0dc service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:06.418809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1334/2645] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:06.690397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d05bce-6590-4c01-9c8f-32ca310a6ab7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:06.692470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42d05bce-6590-4c01-9c8f-32ca310a6ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:06.706960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d05bce-6590-4c01-9c8f-32ca310a6ab7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:06.711641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:06.712518 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:06.712804 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:06.713084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] PATH_INFO: `/v3/projects/fdc656383fed4892b7b50b2bb1a68354` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:06.714288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc656383fed4892b7b50b2bb1a68354)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:06.716374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:06.773026 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc656383fed4892b7b50b2bb1a68354', 'request_id': 'req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.780425 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc656383fed4892b7b50b2bb1a68354', 'request_id': 'req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.787484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc656383fed4892b7b50b2bb1a68354', 'request_id': 'req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.797824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] The token cache is being invalidate because project fdc656383fed4892b7b50b2bb1a68354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:06.798106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.798728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.800901 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1312/2646] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:06 2022] DELETE /identity/v3/projects/fdc656383fed4892b7b50b2bb1a68354 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:06.804913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42b966-e35a-46d2-9218-fe97c781c0dc service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:06.814299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42b966-e35a-46d2-9218-fe97c781c0dc service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:06.873494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:06.874278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:06.874553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:06.874823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] PATH_INFO: `/v3/users/ad663679ba4140e1a618c9bce151b889` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:06.876022 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad663679ba4140e1a618c9bce151b889)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:06.878068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:06.882785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:06.926910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad663679ba4140e1a618c9bce151b889'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.927583 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad663679ba4140e1a618c9bce151b889'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.934480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad663679ba4140e1a618c9bce151b889'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.947115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad663679ba4140e1a618c9bce151b889'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.947829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1186f9c2-baf6-4f99-bb02-3baa6aa24ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad663679ba4140e1a618c9bce151b889'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:06.963249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1335/2647] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:06 2022] DELETE /identity/v3/users/ad663679ba4140e1a618c9bce151b889 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:07.267400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:07.269608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:07.287277 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0622dd-2b65-4aa5-8091-9a7d9e2bc620 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:07.355998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:07.356882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.357168 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.357447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] PATH_INFO: `/v3/projects/16d47512828447798aa866017c3aa83b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.358649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d47512828447798aa866017c3aa83b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:07.360617 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:07.411315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d47512828447798aa866017c3aa83b', 'request_id': 'req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:07.419988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d47512828447798aa866017c3aa83b', 'request_id': 'req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:07.423072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:07.431331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d47512828447798aa866017c3aa83b', 'request_id': 'req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:07.447632 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] The token cache is being invalidate because project 16d47512828447798aa866017c3aa83b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:07.449946 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:07.450966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9b6dc5f8-cdfc-45ee-997d-558bfad8d8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:07.453733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1313/2648] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:07 2022] DELETE /identity/v3/projects/16d47512828447798aa866017c3aa83b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:07.459655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c35c-a2e8-451d-baa5-d444feb6f7b1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.459976 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c35c-a2e8-451d-baa5-d444feb6f7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.460334 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c35c-a2e8-451d-baa5-d444feb6f7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.481462 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3f42c35c-a2e8-451d-baa5-d444feb6f7b1 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:07.483719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3f42c35c-a2e8-451d-baa5-d444feb6f7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:07.577649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1336/2649] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:37:07.584756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378b0a5-10c6-4613-ae23-ee915e59e11d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.585041 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378b0a5-10c6-4613-ae23-ee915e59e11d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.585519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378b0a5-10c6-4613-ae23-ee915e59e11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.607641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a378b0a5-10c6-4613-ae23-ee915e59e11d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:07.608883 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a378b0a5-10c6-4613-ae23-ee915e59e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:07.617703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1314/2650] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:37:07.622081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42c35c-a2e8-451d-baa5-d444feb6f7b1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:07.635025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42c35c-a2e8-451d-baa5-d444feb6f7b1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:07.642818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:07.643313 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.643625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.643913 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.644655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:37:07.644987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:07.646608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:07.662551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1337/2651] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:37:07.665209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378b0a5-10c6-4613-ae23-ee915e59e11d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:07.677990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378b0a5-10c6-4613-ae23-ee915e59e11d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:07.682910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674d924-d8e9-4e42-9f6a-56a97ccfb76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:07.683614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d924-d8e9-4e42-9f6a-56a97ccfb76c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.683932 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d924-d8e9-4e42-9f6a-56a97ccfb76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.684216 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d924-d8e9-4e42-9f6a-56a97ccfb76c admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.684927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674d924-d8e9-4e42-9f6a-56a97ccfb76c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:07.686961 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674d924-d8e9-4e42-9f6a-56a97ccfb76c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:07.720956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1315/2652] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:07 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:07.722434 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:07.733057 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24def-ffc3-42c6-b6c2-afa96a86d30e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:07.802862 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df8dc-e2fd-4788-aa13-4faef60cc5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:07.803558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df8dc-e2fd-4788-aa13-4faef60cc5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.803843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df8dc-e2fd-4788-aa13-4faef60cc5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.804118 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df8dc-e2fd-4788-aa13-4faef60cc5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.804851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42df8dc-e2fd-4788-aa13-4faef60cc5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:07.806769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42df8dc-e2fd-4788-aa13-4faef60cc5f2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:07.821543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:07.830438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1338/2653] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:07 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:07.832143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674d924-d8e9-4e42-9f6a-56a97ccfb76c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:07.842238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674d924-d8e9-4e42-9f6a-56a97ccfb76c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:07.848040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea548e5-4252-478f-8e9d-f85da0c726f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:07.848707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea548e5-4252-478f-8e9d-f85da0c726f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.848982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea548e5-4252-478f-8e9d-f85da0c726f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.849256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea548e5-4252-478f-8e9d-f85da0c726f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.850253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea548e5-4252-478f-8e9d-f85da0c726f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:07.851868 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea548e5-4252-478f-8e9d-f85da0c726f0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:07.862357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1316/2654] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:07.866853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df8dc-e2fd-4788-aa13-4faef60cc5f2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:07.876933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df8dc-e2fd-4788-aa13-4faef60cc5f2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:07.882049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4261e-c64e-4339-a54d-a94a977dbe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:07.882708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4261e-c64e-4339-a54d-a94a977dbe00 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.882995 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4261e-c64e-4339-a54d-a94a977dbe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.883272 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4261e-c64e-4339-a54d-a94a977dbe00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.884029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4261e-c64e-4339-a54d-a94a977dbe00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:07.885710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4261e-c64e-4339-a54d-a94a977dbe00 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:07.896747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1339/2655] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:07.900975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea548e5-4252-478f-8e9d-f85da0c726f0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:07.911557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea548e5-4252-478f-8e9d-f85da0c726f0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:07.916131 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80911df-6a71-48c1-9508-f3fb99a279a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:07.917004 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80911df-6a71-48c1-9508-f3fb99a279a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.917291 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80911df-6a71-48c1-9508-f3fb99a279a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.917568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80911df-6a71-48c1-9508-f3fb99a279a5 admin admin] PATH_INFO: `/v3/projects/8297bf6ad1d44eb8a5c8dc81b42758af/users/66c4886fa2a741e091b9e4bbea908976/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.931730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80911df-6a71-48c1-9508-f3fb99a279a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8297bf6ad1d44eb8a5c8dc81b42758af, user_id=66c4886fa2a741e091b9e4bbea908976, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:07.934366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80911df-6a71-48c1-9508-f3fb99a279a5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:07.947048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1317/2656] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:07 2022] PUT /identity/v3/projects/8297bf6ad1d44eb8a5c8dc81b42758af/users/66c4886fa2a741e091b9e4bbea908976/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:37:07.958992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba75ca-424f-4902-aa00-22858b2ebf0e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:07.959286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba75ca-424f-4902-aa00-22858b2ebf0e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:07.959599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba75ca-424f-4902-aa00-22858b2ebf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:07.986256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bdba75ca-424f-4902-aa00-22858b2ebf0e None None] MFA Rules not processed for user `66c4886fa2a741e091b9e4bbea908976`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:07.987512 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bdba75ca-424f-4902-aa00-22858b2ebf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:08.059058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1340/2657] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:08.063623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80911df-6a71-48c1-9508-f3fb99a279a5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:08.132368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80911df-6a71-48c1-9508-f3fb99a279a5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:08.137086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7cecd-2a8a-4e7c-8c7f-c58e6a017dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:08.137693 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7cecd-2a8a-4e7c-8c7f-c58e6a017dca admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:08.137977 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7cecd-2a8a-4e7c-8c7f-c58e6a017dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:08.138256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7cecd-2a8a-4e7c-8c7f-c58e6a017dca admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:08.138947 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7cecd-2a8a-4e7c-8c7f-c58e6a017dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:08.140893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7cecd-2a8a-4e7c-8c7f-c58e6a017dca admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:08.166064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1318/2658] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:08 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:37:08.170095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba75ca-424f-4902-aa00-22858b2ebf0e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:08.181430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba75ca-424f-4902-aa00-22858b2ebf0e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:08.237106 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c953-7c35-4848-a72c-5be5aed4fdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:08.237768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c953-7c35-4848-a72c-5be5aed4fdd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:08.238048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c953-7c35-4848-a72c-5be5aed4fdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:08.238346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c953-7c35-4848-a72c-5be5aed4fdd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:08.239044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29c953-7c35-4848-a72c-5be5aed4fdd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:08.240987 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29c953-7c35-4848-a72c-5be5aed4fdd8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:08.261005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1341/2659] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:08 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:08.264917 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7cecd-2a8a-4e7c-8c7f-c58e6a017dca admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:08.276137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7cecd-2a8a-4e7c-8c7f-c58e6a017dca admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:08.280709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb042fe-cbc7-4cf2-a830-51e352aee164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:08.281362 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb042fe-cbc7-4cf2-a830-51e352aee164 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:08.281645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb042fe-cbc7-4cf2-a830-51e352aee164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:08.281922 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb042fe-cbc7-4cf2-a830-51e352aee164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:08.282645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb042fe-cbc7-4cf2-a830-51e352aee164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:08.284414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb042fe-cbc7-4cf2-a830-51e352aee164 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:08.296011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1319/2660] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:08.300569 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c953-7c35-4848-a72c-5be5aed4fdd8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:08.310560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c953-7c35-4848-a72c-5be5aed4fdd8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:08.315708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51c80-b0f5-4852-8462-6a74e38e50e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:08.316579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51c80-b0f5-4852-8462-6a74e38e50e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:08.316861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51c80-b0f5-4852-8462-6a74e38e50e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:08.317142 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51c80-b0f5-4852-8462-6a74e38e50e6 admin admin] PATH_INFO: `/v3/projects/1ad49d09097b461eb73a876c9b5e61bc/users/de696fce566d4ccba8751e91ded620dc/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:08.333616 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51c80-b0f5-4852-8462-6a74e38e50e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad49d09097b461eb73a876c9b5e61bc, user_id=de696fce566d4ccba8751e91ded620dc, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:08.336402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51c80-b0f5-4852-8462-6a74e38e50e6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:08.349735 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1342/2661] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:08 2022] PUT /identity/v3/projects/1ad49d09097b461eb73a876c9b5e61bc/users/de696fce566d4ccba8751e91ded620dc/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:08.353861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb042fe-cbc7-4cf2-a830-51e352aee164 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:08.363903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb042fe-cbc7-4cf2-a830-51e352aee164 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:08.416821 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:08.424007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f75bd4-8457-4f8a-ba36-22fe713398fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:08.424705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75bd4-8457-4f8a-ba36-22fe713398fc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:08.425108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75bd4-8457-4f8a-ba36-22fe713398fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:08.425459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75bd4-8457-4f8a-ba36-22fe713398fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:08.426584 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f75bd4-8457-4f8a-ba36-22fe713398fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:08.428619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f75bd4-8457-4f8a-ba36-22fe713398fc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:08.441643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1320/2662] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:08.444240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51c80-b0f5-4852-8462-6a74e38e50e6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:08.454977 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51c80-b0f5-4852-8462-6a74e38e50e6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:08.460189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:08.461071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:08.461353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:08.461630 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] PATH_INFO: `/v3/projects/1ad49d09097b461eb73a876c9b5e61bc/users/de696fce566d4ccba8751e91ded620dc/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:08.463613 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad49d09097b461eb73a876c9b5e61bc, user_id=de696fce566d4ccba8751e91ded620dc, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:08.466251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:08.476760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1343/2663] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:08 2022] PUT /identity/v3/projects/1ad49d09097b461eb73a876c9b5e61bc/users/de696fce566d4ccba8751e91ded620dc/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:08.486633 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddabf3-b911-4f71-86e5-602b32079b89 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:08.487614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddabf3-b911-4f71-86e5-602b32079b89 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:08.488128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddabf3-b911-4f71-86e5-602b32079b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:08.514542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0fddabf3-b911-4f71-86e5-602b32079b89 None None] MFA Rules not processed for user `de696fce566d4ccba8751e91ded620dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:08.515801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0fddabf3-b911-4f71-86e5-602b32079b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:08.582024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1321/2664] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:08.593745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:08.595195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:08.630689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:08.655984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2123de-1bed-42cc-b110-a400d7ea8d5f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:08.660025 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b709ff-b6f3-4811-91c0-56057fa76090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:08.660673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b709ff-b6f3-4811-91c0-56057fa76090 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:08.660957 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b709ff-b6f3-4811-91c0-56057fa76090 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:08.661236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b709ff-b6f3-4811-91c0-56057fa76090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:08.726021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b709ff-b6f3-4811-91c0-56057fa76090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:08.727721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b709ff-b6f3-4811-91c0-56057fa76090 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:08.738199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1344/2665] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:18.579032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=37 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:18.606155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddabf3-b911-4f71-86e5-602b32079b89 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:18.667738 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddabf3-b911-4f71-86e5-602b32079b89 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:18.672742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:18.673636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:18.673911 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:18.674191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] PATH_INFO: `/v3/users/66c4886fa2a741e091b9e4bbea908976` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:18.675478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=66c4886fa2a741e091b9e4bbea908976)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:18.677412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:18.725240 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66c4886fa2a741e091b9e4bbea908976'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:18.733000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66c4886fa2a741e091b9e4bbea908976'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:18.733160 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=34 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:18.745030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66c4886fa2a741e091b9e4bbea908976'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:18.745677 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66c4886fa2a741e091b9e4bbea908976'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:18.753679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66c4886fa2a741e091b9e4bbea908976'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:18.759641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1322/2666] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:18 2022] DELETE /identity/v3/users/66c4886fa2a741e091b9e4bbea908976 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:18.764060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bfd56-4c8c-4236-8021-f62e332affb2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:18.764346 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bfd56-4c8c-4236-8021-f62e332affb2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:18.764623 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bfd56-4c8c-4236-8021-f62e332affb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:18.781782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-740bfd56-4c8c-4236-8021-f62e332affb2 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:18.782961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-740bfd56-4c8c-4236-8021-f62e332affb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:18.845747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1345/2667] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:18.855870 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:18.857314 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:18.914963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a83037-6862-47d3-ae31-cb620396a2fa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:18.918547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0d6db-5596-4985-889a-be622cd09ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:18.919189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d6db-5596-4985-889a-be622cd09ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:18.919498 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d6db-5596-4985-889a-be622cd09ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:18.919799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d6db-5596-4985-889a-be622cd09ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:18.929196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0d6db-5596-4985-889a-be622cd09ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:18.930610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0d6db-5596-4985-889a-be622cd09ca8 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:18.939743 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1323/2668] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:19.210458 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bfd56-4c8c-4236-8021-f62e332affb2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:19.211898 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-740bfd56-4c8c-4236-8021-f62e332affb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:19.224302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bfd56-4c8c-4236-8021-f62e332affb2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:19.228836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:19.229765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:19.230041 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:19.230312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] PATH_INFO: `/v3/projects/8297bf6ad1d44eb8a5c8dc81b42758af` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:19.231536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8297bf6ad1d44eb8a5c8dc81b42758af)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:19.233515 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:19.282419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8297bf6ad1d44eb8a5c8dc81b42758af', 'request_id': 'req-104c93ab-2148-43da-8818-9e53091dc17a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.288803 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8297bf6ad1d44eb8a5c8dc81b42758af', 'request_id': 'req-104c93ab-2148-43da-8818-9e53091dc17a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.294269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8297bf6ad1d44eb8a5c8dc81b42758af', 'request_id': 'req-104c93ab-2148-43da-8818-9e53091dc17a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.309532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] The token cache is being invalidate because project 8297bf6ad1d44eb8a5c8dc81b42758af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:19.309823 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.310532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.312034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1346/2669] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:19 2022] DELETE /identity/v3/projects/8297bf6ad1d44eb8a5c8dc81b42758af => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:19.316771 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0d6db-5596-4985-889a-be622cd09ca8 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:19.326331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0d6db-5596-4985-889a-be622cd09ca8 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:19.344926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:19.386836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:19.387703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:19.387993 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:19.388273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] PATH_INFO: `/v3/users/de696fce566d4ccba8751e91ded620dc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:19.389486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de696fce566d4ccba8751e91ded620dc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:19.391484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:19.438122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de696fce566d4ccba8751e91ded620dc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.449020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de696fce566d4ccba8751e91ded620dc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.462571 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de696fce566d4ccba8751e91ded620dc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.463302 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de696fce566d4ccba8751e91ded620dc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.471983 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2ec22075-3d85-4540-b0e0-4ea1788d6328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de696fce566d4ccba8751e91ded620dc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.479140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1324/2670] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:19 2022] DELETE /identity/v3/users/de696fce566d4ccba8751e91ded620dc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:19.763216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:19.765437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:19.781476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c93ab-2148-43da-8818-9e53091dc17a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:19.842916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:19.843844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:19.844124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:19.844396 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] PATH_INFO: `/v3/projects/1ad49d09097b461eb73a876c9b5e61bc` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:19.845600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad49d09097b461eb73a876c9b5e61bc)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:19.847755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:19.884525 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:19.903901 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad49d09097b461eb73a876c9b5e61bc', 'request_id': 'req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.910832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad49d09097b461eb73a876c9b5e61bc', 'request_id': 'req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.917094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad49d09097b461eb73a876c9b5e61bc', 'request_id': 'req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.932205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] The token cache is being invalidate because project 1ad49d09097b461eb73a876c9b5e61bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:19.932577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.933293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-291e99ef-fdd2-4e5e-8b7e-2f530bb65523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:19.935129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1347/2671] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:19 2022] DELETE /identity/v3/projects/1ad49d09097b461eb73a876c9b5e61bc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:19.939892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224987-bc43-47a2-a716-a2c6d6949c8f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:19.942108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224987-bc43-47a2-a716-a2c6d6949c8f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:19.942406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224987-bc43-47a2-a716-a2c6d6949c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:19.961845 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4b224987-bc43-47a2-a716-a2c6d6949c8f None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:19.963110 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4b224987-bc43-47a2-a716-a2c6d6949c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:20.028011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1325/2672] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:37:20.036943 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e01f7-1752-4a95-aba2-204ba3f7d8ac None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.037329 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e01f7-1752-4a95-aba2-204ba3f7d8ac None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.037670 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e01f7-1752-4a95-aba2-204ba3f7d8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.056505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ae0e01f7-1752-4a95-aba2-204ba3f7d8ac None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:20.057682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ae0e01f7-1752-4a95-aba2-204ba3f7d8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:20.066859 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1348/2673] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:20.071086 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224987-bc43-47a2-a716-a2c6d6949c8f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.080705 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224987-bc43-47a2-a716-a2c6d6949c8f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.085356 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.086096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.086381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.086656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.087435 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:37:20.087748 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.089367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.101334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1326/2674] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:37:20.105705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e01f7-1752-4a95-aba2-204ba3f7d8ac None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.115404 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e01f7-1752-4a95-aba2-204ba3f7d8ac None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.120397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef3a1c-e76d-4aec-81e4-1190cdfdf38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.121033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef3a1c-e76d-4aec-81e4-1190cdfdf38a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.121308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef3a1c-e76d-4aec-81e4-1190cdfdf38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.121577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef3a1c-e76d-4aec-81e4-1190cdfdf38a admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.122256 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef3a1c-e76d-4aec-81e4-1190cdfdf38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.124179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef3a1c-e76d-4aec-81e4-1190cdfdf38a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.149188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1349/2675] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:20.153245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.163925 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8a3a-1340-406c-b097-48ffdfabe720 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.189987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:20.225350 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7dee4-9f5e-4639-944c-e0b7cd727e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.226021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7dee4-9f5e-4639-944c-e0b7cd727e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.226307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7dee4-9f5e-4639-944c-e0b7cd727e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.226586 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7dee4-9f5e-4639-944c-e0b7cd727e84 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.227274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7dee4-9f5e-4639-944c-e0b7cd727e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.229275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7dee4-9f5e-4639-944c-e0b7cd727e84 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.248365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1327/2676] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:20.252440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef3a1c-e76d-4aec-81e4-1190cdfdf38a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.263425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef3a1c-e76d-4aec-81e4-1190cdfdf38a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.268229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14792-c0b4-4245-8f68-02ef1b24be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.269427 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14792-c0b4-4245-8f68-02ef1b24be6b admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.269701 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14792-c0b4-4245-8f68-02ef1b24be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.270004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14792-c0b4-4245-8f68-02ef1b24be6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.270728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d14792-c0b4-4245-8f68-02ef1b24be6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.272362 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d14792-c0b4-4245-8f68-02ef1b24be6b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.282579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1350/2677] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:37:20.286354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7dee4-9f5e-4639-944c-e0b7cd727e84 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.296138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7dee4-9f5e-4639-944c-e0b7cd727e84 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.300824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cbc41-bdfd-4ff2-af30-e66df3540ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.301494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cbc41-bdfd-4ff2-af30-e66df3540ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.301776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cbc41-bdfd-4ff2-af30-e66df3540ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.302072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cbc41-bdfd-4ff2-af30-e66df3540ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.302778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cbc41-bdfd-4ff2-af30-e66df3540ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.304387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cbc41-bdfd-4ff2-af30-e66df3540ab8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.314227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1328/2678] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:20.318933 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14792-c0b4-4245-8f68-02ef1b24be6b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.329215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d14792-c0b4-4245-8f68-02ef1b24be6b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.334601 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d941aba-a74e-47d6-9801-c3c89c0704a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.335509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941aba-a74e-47d6-9801-c3c89c0704a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.335794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941aba-a74e-47d6-9801-c3c89c0704a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.336073 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941aba-a74e-47d6-9801-c3c89c0704a3 admin admin] PATH_INFO: `/v3/projects/483e444b97ea4015b2b0faea47d73968/users/f07325c318d94aecab55ff9809275fe9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.350611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d941aba-a74e-47d6-9801-c3c89c0704a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483e444b97ea4015b2b0faea47d73968, user_id=f07325c318d94aecab55ff9809275fe9, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.354348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d941aba-a74e-47d6-9801-c3c89c0704a3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.367785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1351/2679] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:20 2022] PUT /identity/v3/projects/483e444b97ea4015b2b0faea47d73968/users/f07325c318d94aecab55ff9809275fe9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:20.379447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70698f64-3aa8-4573-a7aa-c35b0e2b6342 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.379742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70698f64-3aa8-4573-a7aa-c35b0e2b6342 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.380016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70698f64-3aa8-4573-a7aa-c35b0e2b6342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.406984 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-70698f64-3aa8-4573-a7aa-c35b0e2b6342 None None] MFA Rules not processed for user `f07325c318d94aecab55ff9809275fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:20.408258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70698f64-3aa8-4573-a7aa-c35b0e2b6342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:20.460889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6bf02-1764-4293-a992-4e7a13cc661b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.461277 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6bf02-1764-4293-a992-4e7a13cc661b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.461642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6bf02-1764-4293-a992-4e7a13cc661b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.476063 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1329/2680] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:20.479398 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-19d6bf02-1764-4293-a992-4e7a13cc661b None None] MFA Rules not processed for user `4a827bedc4b84508bb7770a09efee3f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:20.480378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70698f64-3aa8-4573-a7aa-c35b0e2b6342 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.480608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-19d6bf02-1764-4293-a992-4e7a13cc661b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:20.543581 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70698f64-3aa8-4573-a7aa-c35b0e2b6342 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.545199 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1352/2681] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:20.548245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117d4f1-0474-4e94-96ee-1e687a4a3c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.548905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d4f1-0474-4e94-96ee-1e687a4a3c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.549522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d4f1-0474-4e94-96ee-1e687a4a3c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.549864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d4f1-0474-4e94-96ee-1e687a4a3c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.550563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117d4f1-0474-4e94-96ee-1e687a4a3c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.552486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117d4f1-0474-4e94-96ee-1e687a4a3c1f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.587405 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1330/2682] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:20.589523 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6bf02-1764-4293-a992-4e7a13cc661b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.602925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6bf02-1764-4293-a992-4e7a13cc661b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.671916 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:20.684317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f37f7-0d4f-4a9a-9efc-d4a66d22a16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.685060 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f37f7-0d4f-4a9a-9efc-d4a66d22a16c admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.685382 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f37f7-0d4f-4a9a-9efc-d4a66d22a16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.685666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f37f7-0d4f-4a9a-9efc-d4a66d22a16c admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.686505 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f37f7-0d4f-4a9a-9efc-d4a66d22a16c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.688815 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f37f7-0d4f-4a9a-9efc-d4a66d22a16c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.712312 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1353/2683] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:20.716007 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117d4f1-0474-4e94-96ee-1e687a4a3c1f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.728273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117d4f1-0474-4e94-96ee-1e687a4a3c1f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.733163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b316-4cf9-4e12-98bb-6697b7075f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.733832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b316-4cf9-4e12-98bb-6697b7075f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.734114 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b316-4cf9-4e12-98bb-6697b7075f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.734393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b316-4cf9-4e12-98bb-6697b7075f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.735121 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881b316-4cf9-4e12-98bb-6697b7075f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.736736 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881b316-4cf9-4e12-98bb-6697b7075f08 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.748289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1331/2684] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:20.752267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f37f7-0d4f-4a9a-9efc-d4a66d22a16c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.763593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f37f7-0d4f-4a9a-9efc-d4a66d22a16c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.769201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62439cf-8208-4f31-864e-99f0d506be49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.769877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62439cf-8208-4f31-864e-99f0d506be49 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.770163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62439cf-8208-4f31-864e-99f0d506be49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.770437 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62439cf-8208-4f31-864e-99f0d506be49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.771177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62439cf-8208-4f31-864e-99f0d506be49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.772818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62439cf-8208-4f31-864e-99f0d506be49 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.783066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1354/2685] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:37:20.789073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b316-4cf9-4e12-98bb-6697b7075f08 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.802424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b316-4cf9-4e12-98bb-6697b7075f08 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.809556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.811018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.811307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.811622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] PATH_INFO: `/v3/projects/68797f33c80e4c03ac36b70cc3a0dbe4/users/8a826bbaa3fe4fc884bef96da632771a/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.823928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:20.833478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=68797f33c80e4c03ac36b70cc3a0dbe4, user_id=8a826bbaa3fe4fc884bef96da632771a, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.836295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.850070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1332/2686] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:20 2022] PUT /identity/v3/projects/68797f33c80e4c03ac36b70cc3a0dbe4/users/8a826bbaa3fe4fc884bef96da632771a/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:20.860832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415a8b2-f682-499d-ae86-b2eae4a514fe None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.861126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415a8b2-f682-499d-ae86-b2eae4a514fe None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.861406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415a8b2-f682-499d-ae86-b2eae4a514fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.887826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.889550 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2415a8b2-f682-499d-ae86-b2eae4a514fe None None] MFA Rules not processed for user `8a826bbaa3fe4fc884bef96da632771a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:20.889691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:20.890750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2415a8b2-f682-499d-ae86-b2eae4a514fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:20.953728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ef64-111d-4e49-895f-fcbae329d0ab admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:20.958154 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:20.959070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:20.959526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:20.959861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:20.961302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1355/2687] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:20.962671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:20.965838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415a8b2-f682-499d-ae86-b2eae4a514fe None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:20.975647 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:20.977099 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:20.985288 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1333/2688] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:37:21.035478 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415a8b2-f682-499d-ae86-b2eae4a514fe None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.040500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a49b0-d1e8-486a-ae70-c749fe4b96f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.041960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a49b0-d1e8-486a-ae70-c749fe4b96f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.041960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a49b0-d1e8-486a-ae70-c749fe4b96f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.041960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a49b0-d1e8-486a-ae70-c749fe4b96f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.042455 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a49b0-d1e8-486a-ae70-c749fe4b96f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.044403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a49b0-d1e8-486a-ae70-c749fe4b96f3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.071918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1356/2689] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:20 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:37:21.076106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:21.088065 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c5c4a-e04f-4f92-b388-e5a9e4705efd service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.145258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93116c6-5519-46e6-8eb0-a251c7ef69a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.145969 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93116c6-5519-46e6-8eb0-a251c7ef69a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.146513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93116c6-5519-46e6-8eb0-a251c7ef69a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.146824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93116c6-5519-46e6-8eb0-a251c7ef69a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.147723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93116c6-5519-46e6-8eb0-a251c7ef69a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.149777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93116c6-5519-46e6-8eb0-a251c7ef69a4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.170447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1334/2690] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:21 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:21.174309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a49b0-d1e8-486a-ae70-c749fe4b96f3 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:21.185421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a49b0-d1e8-486a-ae70-c749fe4b96f3 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.190711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5d030-e12a-408f-bc55-9f765f9c6444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.191411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5d030-e12a-408f-bc55-9f765f9c6444 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.191693 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5d030-e12a-408f-bc55-9f765f9c6444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.191965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5d030-e12a-408f-bc55-9f765f9c6444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.192698 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5d030-e12a-408f-bc55-9f765f9c6444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.194295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5d030-e12a-408f-bc55-9f765f9c6444 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.205014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1357/2691] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:21.209708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93116c6-5519-46e6-8eb0-a251c7ef69a4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:21.220111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93116c6-5519-46e6-8eb0-a251c7ef69a4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.225382 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed05f3-99a6-4ab3-bc49-697a3fbe66ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.226275 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed05f3-99a6-4ab3-bc49-697a3fbe66ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.226559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed05f3-99a6-4ab3-bc49-697a3fbe66ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.226837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed05f3-99a6-4ab3-bc49-697a3fbe66ae admin admin] PATH_INFO: `/v3/projects/d8b0093b7404461eb5128b1c99725879/users/950516c87c1c42c3a35d7ea7c1dbb4ee/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.242167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed05f3-99a6-4ab3-bc49-697a3fbe66ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b0093b7404461eb5128b1c99725879, user_id=950516c87c1c42c3a35d7ea7c1dbb4ee, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.245178 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed05f3-99a6-4ab3-bc49-697a3fbe66ae admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.259483 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1335/2692] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:21 2022] PUT /identity/v3/projects/d8b0093b7404461eb5128b1c99725879/users/950516c87c1c42c3a35d7ea7c1dbb4ee/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:21.263551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5d030-e12a-408f-bc55-9f765f9c6444 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:21.273848 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5d030-e12a-408f-bc55-9f765f9c6444 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.312886 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:21.340035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab24ec-624e-443f-b7eb-ee5ac4b6fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.340727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab24ec-624e-443f-b7eb-ee5ac4b6fbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.341007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab24ec-624e-443f-b7eb-ee5ac4b6fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.341279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab24ec-624e-443f-b7eb-ee5ac4b6fbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.342001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab24ec-624e-443f-b7eb-ee5ac4b6fbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.343724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab24ec-624e-443f-b7eb-ee5ac4b6fbe9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.354058 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1358/2693] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:21.358464 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed05f3-99a6-4ab3-bc49-697a3fbe66ae admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:21.369787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed05f3-99a6-4ab3-bc49-697a3fbe66ae admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.374878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.375895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.376171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.376578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] PATH_INFO: `/v3/projects/d8b0093b7404461eb5128b1c99725879/users/950516c87c1c42c3a35d7ea7c1dbb4ee/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.379537 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b0093b7404461eb5128b1c99725879, user_id=950516c87c1c42c3a35d7ea7c1dbb4ee, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.382387 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.393296 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1336/2694] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:21 2022] PUT /identity/v3/projects/d8b0093b7404461eb5128b1c99725879/users/950516c87c1c42c3a35d7ea7c1dbb4ee/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:37:21.403889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf9a8-fa2f-43f4-b216-17f6b690769b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.404174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf9a8-fa2f-43f4-b216-17f6b690769b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.404449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf9a8-fa2f-43f4-b216-17f6b690769b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.432368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-199cf9a8-fa2f-43f4-b216-17f6b690769b None None] MFA Rules not processed for user `950516c87c1c42c3a35d7ea7c1dbb4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:21.433579 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-199cf9a8-fa2f-43f4-b216-17f6b690769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:21.472364 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:21.475053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:21.515769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1359/2695] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:21.528664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cf9a8-fa2f-43f4-b216-17f6b690769b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:21.530326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-199cf9a8-fa2f-43f4-b216-17f6b690769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:21.543412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:21.569123 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4931cf7-048a-4264-b445-53847ed0b67b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.574172 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e13fa-5278-478b-babf-262797ab22de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.574825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e13fa-5278-478b-babf-262797ab22de service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.575111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e13fa-5278-478b-babf-262797ab22de service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.575421 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e13fa-5278-478b-babf-262797ab22de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.577274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-560e13fa-5278-478b-babf-262797ab22de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:21.589464 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cf9a8-fa2f-43f4-b216-17f6b690769b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.591902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e13fa-5278-478b-babf-262797ab22de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.593371 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae259ca-e7d1-4df5-8029-5867d5153d6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.593573 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e13fa-5278-478b-babf-262797ab22de service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.594014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae259ca-e7d1-4df5-8029-5867d5153d6d service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.594292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae259ca-e7d1-4df5-8029-5867d5153d6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.594567 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae259ca-e7d1-4df5-8029-5867d5153d6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.604321 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1337/2696] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:37:21.651534 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae259ca-e7d1-4df5-8029-5867d5153d6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.652999 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae259ca-e7d1-4df5-8029-5867d5153d6d service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.662476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1360/2697] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:21.728286 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e13fa-5278-478b-babf-262797ab22de service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:21.731404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-560e13fa-5278-478b-babf-262797ab22de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:21.807753 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e13fa-5278-478b-babf-262797ab22de service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:21.817805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:21.820156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.820517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.820900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] PATH_INFO: `/v3/users/0afd07ccf7904dc0978b774cfc877c74` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.822837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afd07ccf7904dc0978b774cfc877c74)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:21.825781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:21.891955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afd07ccf7904dc0978b774cfc877c74'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:21.900627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afd07ccf7904dc0978b774cfc877c74'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:21.916666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afd07ccf7904dc0978b774cfc877c74'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:21.917648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afd07ccf7904dc0978b774cfc877c74'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:21.928087 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afd07ccf7904dc0978b774cfc877c74'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:21.936013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1338/2698] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:21 2022] DELETE /identity/v3/users/0afd07ccf7904dc0978b774cfc877c74 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:21.940595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be4152-2871-41ae-bb2a-3ce7539f124a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:21.941271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be4152-2871-41ae-bb2a-3ce7539f124a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:21.941271 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be4152-2871-41ae-bb2a-3ce7539f124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:21.959015 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b7be4152-2871-41ae-bb2a-3ce7539f124a None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:21.960340 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b7be4152-2871-41ae-bb2a-3ce7539f124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:21.969877 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:22.035990 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1361/2699] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:37:22.045808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:22.047287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:22.112833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900c12c-e380-4ff3-b8bc-45ffb54e84af admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:22.116619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04568767-f728-4519-9982-04714475f029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:22.117308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568767-f728-4519-9982-04714475f029 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.117592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568767-f728-4519-9982-04714475f029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.117872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568767-f728-4519-9982-04714475f029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.124656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:22.130642 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04568767-f728-4519-9982-04714475f029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:22.132601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04568767-f728-4519-9982-04714475f029 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:22.143805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1339/2700] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:22.416380 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be4152-2871-41ae-bb2a-3ce7539f124a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:22.417837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b7be4152-2871-41ae-bb2a-3ce7539f124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:22.430932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be4152-2871-41ae-bb2a-3ce7539f124a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:22.435686 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:22.436662 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.437005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.437348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] PATH_INFO: `/v3/projects/16275a7776744f2a9ddd4016e626c201` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.438628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16275a7776744f2a9ddd4016e626c201)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:22.440879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:22.494780 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16275a7776744f2a9ddd4016e626c201', 'request_id': 'req-63ce2314-2451-49c4-b584-22b9678fdc48'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:22.502303 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16275a7776744f2a9ddd4016e626c201', 'request_id': 'req-63ce2314-2451-49c4-b584-22b9678fdc48'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:22.508292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16275a7776744f2a9ddd4016e626c201', 'request_id': 'req-63ce2314-2451-49c4-b584-22b9678fdc48'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:22.521934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] The token cache is being invalidate because project 16275a7776744f2a9ddd4016e626c201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:22.522224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:22.522865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-63ce2314-2451-49c4-b584-22b9678fdc48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:22.524530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1362/2701] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:22 2022] DELETE /identity/v3/projects/16275a7776744f2a9ddd4016e626c201 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:22.529410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f08cb-297d-4d64-95c5-1f98663c1305 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.529699 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f08cb-297d-4d64-95c5-1f98663c1305 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.529976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f08cb-297d-4d64-95c5-1f98663c1305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.550210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-019f08cb-297d-4d64-95c5-1f98663c1305 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:22.551720 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-019f08cb-297d-4d64-95c5-1f98663c1305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:22.623300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1340/2702] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:22.631958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e44e-acc9-4c59-b7ae-ca7b7c194376 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.632274 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e44e-acc9-4c59-b7ae-ca7b7c194376 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.632565 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e44e-acc9-4c59-b7ae-ca7b7c194376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.660117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-fb87e44e-acc9-4c59-b7ae-ca7b7c194376 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:22.661607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fb87e44e-acc9-4c59-b7ae-ca7b7c194376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:22.671068 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1363/2703] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:22.675774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f08cb-297d-4d64-95c5-1f98663c1305 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:22.686919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f08cb-297d-4d64-95c5-1f98663c1305 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:22.692407 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:22.693184 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.694020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.694020 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.694583 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:37:22.695628 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:22.697319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:22.711383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1341/2704] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:37:22.715722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87e44e-acc9-4c59-b7ae-ca7b7c194376 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:22.729243 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87e44e-acc9-4c59-b7ae-ca7b7c194376 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:22.735890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a66b-fd8d-4bec-b891-52047adc9499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:22.736988 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a66b-fd8d-4bec-b891-52047adc9499 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.737353 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a66b-fd8d-4bec-b891-52047adc9499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.737849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a66b-fd8d-4bec-b891-52047adc9499 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.738639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a66b-fd8d-4bec-b891-52047adc9499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:22.740651 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a66b-fd8d-4bec-b891-52047adc9499 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:22.768864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1364/2705] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:22 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:22.773092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:22.783100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a891811-905e-497d-80f4-6502bb0edfed admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:22.839051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f623c-06a1-4231-939f-8637eede7e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:22.839754 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f623c-06a1-4231-939f-8637eede7e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.840038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f623c-06a1-4231-939f-8637eede7e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.840318 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f623c-06a1-4231-939f-8637eede7e55 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.841006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f623c-06a1-4231-939f-8637eede7e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:22.842914 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f623c-06a1-4231-939f-8637eede7e55 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:22.862657 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1342/2706] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:22 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:37:22.866731 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a66b-fd8d-4bec-b891-52047adc9499 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:22.876917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a66b-fd8d-4bec-b891-52047adc9499 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:22.883620 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2482d-5c59-41de-ab4b-a711dff8f9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:22.884302 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2482d-5c59-41de-ab4b-a711dff8f9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.884581 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2482d-5c59-41de-ab4b-a711dff8f9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.884855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2482d-5c59-41de-ab4b-a711dff8f9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.885666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2482d-5c59-41de-ab4b-a711dff8f9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:22.887275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2482d-5c59-41de-ab4b-a711dff8f9e1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:22.903962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1365/2707] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:22.907786 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f623c-06a1-4231-939f-8637eede7e55 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:22.919843 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f623c-06a1-4231-939f-8637eede7e55 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:22.924857 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a5c59-d81b-4778-96a7-3079e939c8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:22.926156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5c59-d81b-4778-96a7-3079e939c8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.926442 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5c59-d81b-4778-96a7-3079e939c8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.926728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5c59-d81b-4778-96a7-3079e939c8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.927475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a5c59-d81b-4778-96a7-3079e939c8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:22.929051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a5c59-d81b-4778-96a7-3079e939c8d2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:22.940325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1343/2708] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:22.944230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2482d-5c59-41de-ab4b-a711dff8f9e1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:22.953996 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2482d-5c59-41de-ab4b-a711dff8f9e1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:22.959407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:22.960320 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:22.960599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:22.960872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] PATH_INFO: `/v3/projects/0b0cf96d4c8b4e1e96ccde35364e85de/users/732e20214fbe402088ccaf513f664dee/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:22.976535 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0cf96d4c8b4e1e96ccde35364e85de, user_id=732e20214fbe402088ccaf513f664dee, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:22.979202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:22.993897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1366/2709] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:22 2022] PUT /identity/v3/projects/0b0cf96d4c8b4e1e96ccde35364e85de/users/732e20214fbe402088ccaf513f664dee/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:37:23.005649 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2598-695d-453b-8665-6dfc0508eb65 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:23.005881 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2598-695d-453b-8665-6dfc0508eb65 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:23.006164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2598-695d-453b-8665-6dfc0508eb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:23.048478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f74d2598-695d-453b-8665-6dfc0508eb65 None None] MFA Rules not processed for user `732e20214fbe402088ccaf513f664dee`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:23.050132 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f74d2598-695d-453b-8665-6dfc0508eb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:23.109441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:23.144391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1344/2710] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:23.181096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:23.182873 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:23.220887 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:23.246451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cdd01-bd3a-4b21-82ff-866f216bf308 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:23.250140 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28bfca-b152-4d79-ad66-44b7b843d0e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:23.250785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28bfca-b152-4d79-ad66-44b7b843d0e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:23.251066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28bfca-b152-4d79-ad66-44b7b843d0e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:23.251372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28bfca-b152-4d79-ad66-44b7b843d0e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:23.261561 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28bfca-b152-4d79-ad66-44b7b843d0e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:23.262968 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28bfca-b152-4d79-ad66-44b7b843d0e8 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:23.273679 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1367/2711] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:23.559942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d2598-695d-453b-8665-6dfc0508eb65 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:23.572799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d2598-695d-453b-8665-6dfc0508eb65 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:23.577539 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:23.579251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:23.579662 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:23.581527 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:23.584224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:23.685866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:23.688899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:23.703358 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1345/2712] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:37:24.366426 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28bfca-b152-4d79-ad66-44b7b843d0e8 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:24.377908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28bfca-b152-4d79-ad66-44b7b843d0e8 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:24.382531 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3709d8-1bdf-4e12-b079-8dd0e5325a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:24.383420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3709d8-1bdf-4e12-b079-8dd0e5325a30 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:24.383802 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3709d8-1bdf-4e12-b079-8dd0e5325a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:24.384181 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3709d8-1bdf-4e12-b079-8dd0e5325a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:24.386750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8c3709d8-1bdf-4e12-b079-8dd0e5325a30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:24.495718 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3709d8-1bdf-4e12-b079-8dd0e5325a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:24.497248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3709d8-1bdf-4e12-b079-8dd0e5325a30 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:24.508526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1368/2713] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:37:28.363992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:28.365402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:28.429319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201caa-a8ec-4675-8c3c-d470ab1096d4 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:28.433822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:28.434698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:28.434981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:28.435258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] PATH_INFO: `/v3/users/f07325c318d94aecab55ff9809275fe9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:28.436493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07325c318d94aecab55ff9809275fe9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:28.438428 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:28.484826 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07325c318d94aecab55ff9809275fe9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:28.493189 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07325c318d94aecab55ff9809275fe9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:28.507484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07325c318d94aecab55ff9809275fe9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:28.508210 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07325c318d94aecab55ff9809275fe9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:28.515261 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07325c318d94aecab55ff9809275fe9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:28.521778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1346/2714] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:28 2022] DELETE /identity/v3/users/f07325c318d94aecab55ff9809275fe9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:28.526551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86763077-ea6e-4e19-a213-ad07c332ebaa None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:28.527051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86763077-ea6e-4e19-a213-ad07c332ebaa None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:28.527194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86763077-ea6e-4e19-a213-ad07c332ebaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:28.545139 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-86763077-ea6e-4e19-a213-ad07c332ebaa None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:28.546499 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-86763077-ea6e-4e19-a213-ad07c332ebaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:28.607480 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1369/2715] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:37:28.617552 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:28.618960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:28.629100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:28.679359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c5e0a-f535-4acd-8568-99058ccc369c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:28.682992 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09841869-1741-4716-88c2-d9d5f15578f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:28.683681 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09841869-1741-4716-88c2-d9d5f15578f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:28.683966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09841869-1741-4716-88c2-d9d5f15578f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:28.684244 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09841869-1741-4716-88c2-d9d5f15578f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:28.693737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09841869-1741-4716-88c2-d9d5f15578f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:28.695160 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09841869-1741-4716-88c2-d9d5f15578f8 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:28.704441 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1347/2716] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:29.001729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86763077-ea6e-4e19-a213-ad07c332ebaa None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:29.004778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-86763077-ea6e-4e19-a213-ad07c332ebaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:29.022508 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86763077-ea6e-4e19-a213-ad07c332ebaa None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:29.028433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:29.029628 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:29.030111 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:29.030337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] PATH_INFO: `/v3/projects/483e444b97ea4015b2b0faea47d73968` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:29.031789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483e444b97ea4015b2b0faea47d73968)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:29.034589 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:29.087885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483e444b97ea4015b2b0faea47d73968', 'request_id': 'req-14e3aec6-cdba-46f8-ae59-562cde473a44'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.094250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483e444b97ea4015b2b0faea47d73968', 'request_id': 'req-14e3aec6-cdba-46f8-ae59-562cde473a44'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.099864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483e444b97ea4015b2b0faea47d73968', 'request_id': 'req-14e3aec6-cdba-46f8-ae59-562cde473a44'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.102323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:29.114120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] The token cache is being invalidate because project 483e444b97ea4015b2b0faea47d73968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:29.114459 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.115075 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.116658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1370/2717] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:29 2022] DELETE /identity/v3/projects/483e444b97ea4015b2b0faea47d73968 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:37:29.120867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09841869-1741-4716-88c2-d9d5f15578f8 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:29.131381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09841869-1741-4716-88c2-d9d5f15578f8 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:29.187303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:29.188213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:29.188495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:29.188770 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] PATH_INFO: `/v3/users/8a826bbaa3fe4fc884bef96da632771a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:29.190658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a826bbaa3fe4fc884bef96da632771a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:29.192661 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:29.239396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a826bbaa3fe4fc884bef96da632771a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.247634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a826bbaa3fe4fc884bef96da632771a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.260783 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a826bbaa3fe4fc884bef96da632771a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.261414 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a826bbaa3fe4fc884bef96da632771a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.268838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a826bbaa3fe4fc884bef96da632771a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.274665 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1348/2718] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:29 2022] DELETE /identity/v3/users/8a826bbaa3fe4fc884bef96da632771a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:29.547174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:29.548779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:29.566558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3aec6-cdba-46f8-ae59-562cde473a44 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:29.622860 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:29.623682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:29.623961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:29.624234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] PATH_INFO: `/v3/projects/68797f33c80e4c03ac36b70cc3a0dbe4` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:29.625419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68797f33c80e4c03ac36b70cc3a0dbe4)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:29.628530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:29.675021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68797f33c80e4c03ac36b70cc3a0dbe4', 'request_id': 'req-1da481b5-a8f0-4f1f-b085-933042543f18'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.681746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68797f33c80e4c03ac36b70cc3a0dbe4', 'request_id': 'req-1da481b5-a8f0-4f1f-b085-933042543f18'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.688209 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68797f33c80e4c03ac36b70cc3a0dbe4', 'request_id': 'req-1da481b5-a8f0-4f1f-b085-933042543f18'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.702556 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] The token cache is being invalidate because project 68797f33c80e4c03ac36b70cc3a0dbe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:29.702765 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.703402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.704970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1371/2719] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:29 2022] DELETE /identity/v3/projects/68797f33c80e4c03ac36b70cc3a0dbe4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:29.708908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:29.711273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:29.727963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637e8b2-283a-426d-a7fb-88557ca68a61 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:29.757600 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:29.793972 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:29.794867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:29.795149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:29.795463 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] PATH_INFO: `/v3/users/950516c87c1c42c3a35d7ea7c1dbb4ee` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:29.796658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=950516c87c1c42c3a35d7ea7c1dbb4ee)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:29.798608 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:29.855426 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950516c87c1c42c3a35d7ea7c1dbb4ee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.857124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950516c87c1c42c3a35d7ea7c1dbb4ee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.870170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950516c87c1c42c3a35d7ea7c1dbb4ee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.870841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950516c87c1c42c3a35d7ea7c1dbb4ee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.877894 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-789bda2c-00d7-44f3-9731-f34a34d9d2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950516c87c1c42c3a35d7ea7c1dbb4ee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:29.884224 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1349/2720] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:29 2022] DELETE /identity/v3/users/950516c87c1c42c3a35d7ea7c1dbb4ee => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:30.177194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:30.179317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:30.286711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da481b5-a8f0-4f1f-b085-933042543f18 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:30.309938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:30.373007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:30.373007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.373007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.373007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] PATH_INFO: `/v3/projects/d8b0093b7404461eb5128b1c99725879` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.373438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b0093b7404461eb5128b1c99725879)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:30.373438 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:30.427851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b0093b7404461eb5128b1c99725879', 'request_id': 'req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:30.435992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b0093b7404461eb5128b1c99725879', 'request_id': 'req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:30.446937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b0093b7404461eb5128b1c99725879', 'request_id': 'req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:30.458066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] The token cache is being invalidate because project d8b0093b7404461eb5128b1c99725879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:30.458359 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:30.458985 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241bfe4c-e956-4b21-8b72-fd32f1e4f04b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:30.461772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1372/2721] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:30 2022] DELETE /identity/v3/projects/d8b0093b7404461eb5128b1c99725879 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:30.466366 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0945da-bdbd-4f2a-a5f5-0c57a6743f2c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.466658 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0945da-bdbd-4f2a-a5f5-0c57a6743f2c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.466942 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0945da-bdbd-4f2a-a5f5-0c57a6743f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.488460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1f0945da-bdbd-4f2a-a5f5-0c57a6743f2c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:30.489691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f0945da-bdbd-4f2a-a5f5-0c57a6743f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:30.550266 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1350/2722] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:30.611000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad48c8-fea1-4301-8968-1fcf94a0cdf6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.612206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad48c8-fea1-4301-8968-1fcf94a0cdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.612610 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad48c8-fea1-4301-8968-1fcf94a0cdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.633069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-fcad48c8-fea1-4301-8968-1fcf94a0cdf6 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:30.634231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fcad48c8-fea1-4301-8968-1fcf94a0cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:30.642105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1373/2723] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:30.646394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0945da-bdbd-4f2a-a5f5-0c57a6743f2c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:30.656044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0945da-bdbd-4f2a-a5f5-0c57a6743f2c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:30.661186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:30.661838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.662118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.662394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.663124 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:37:30.663470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:30.665051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:30.676982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1351/2724] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:37:30.681260 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad48c8-fea1-4301-8968-1fcf94a0cdf6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:30.690711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad48c8-fea1-4301-8968-1fcf94a0cdf6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:30.695323 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e584bb-6b0e-4946-ad7d-d98863a7634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:30.696024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e584bb-6b0e-4946-ad7d-d98863a7634d admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.696304 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e584bb-6b0e-4946-ad7d-d98863a7634d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.696576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e584bb-6b0e-4946-ad7d-d98863a7634d admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.697261 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e584bb-6b0e-4946-ad7d-d98863a7634d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:30.699153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e584bb-6b0e-4946-ad7d-d98863a7634d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:30.729493 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1374/2725] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:30 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:30.730734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:30.741553 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23457e17-d92a-439b-81cd-ee0cacb05c70 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:30.772948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:30.799944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76702452-7a83-437f-8813-9520cb4c0984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:30.800627 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76702452-7a83-437f-8813-9520cb4c0984 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.800910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76702452-7a83-437f-8813-9520cb4c0984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.801182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76702452-7a83-437f-8813-9520cb4c0984 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.801878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76702452-7a83-437f-8813-9520cb4c0984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:30.803819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76702452-7a83-437f-8813-9520cb4c0984 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:30.826075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1352/2726] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:30 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:30.830094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e584bb-6b0e-4946-ad7d-d98863a7634d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:30.841217 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e584bb-6b0e-4946-ad7d-d98863a7634d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:30.846530 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a281-ab77-424c-a6a7-a609cdfbfa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:30.847775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a281-ab77-424c-a6a7-a609cdfbfa5e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.848045 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a281-ab77-424c-a6a7-a609cdfbfa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.848317 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a281-ab77-424c-a6a7-a609cdfbfa5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.849049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13a281-ab77-424c-a6a7-a609cdfbfa5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:30.850634 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13a281-ab77-424c-a6a7-a609cdfbfa5e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:30.862215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1375/2727] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:37:30.866139 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76702452-7a83-437f-8813-9520cb4c0984 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:30.876352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76702452-7a83-437f-8813-9520cb4c0984 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:30.882113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388bf98-f21c-424b-8c19-23fd08ff8b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:30.882789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388bf98-f21c-424b-8c19-23fd08ff8b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.883072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388bf98-f21c-424b-8c19-23fd08ff8b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.883365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388bf98-f21c-424b-8c19-23fd08ff8b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.884108 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388bf98-f21c-424b-8c19-23fd08ff8b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:30.885697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388bf98-f21c-424b-8c19-23fd08ff8b07 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:30.896822 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1353/2728] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:30.900532 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a281-ab77-424c-a6a7-a609cdfbfa5e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:30.911315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a281-ab77-424c-a6a7-a609cdfbfa5e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:30.916603 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:30.917471 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.917748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.918019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] PATH_INFO: `/v3/projects/924cd779de6f4c608248eae1c02009b4/users/72644a4c8db443658b19af14756daaf0/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.934658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924cd779de6f4c608248eae1c02009b4, user_id=72644a4c8db443658b19af14756daaf0, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:30.937379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:30.953881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1376/2729] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:30 2022] PUT /identity/v3/projects/924cd779de6f4c608248eae1c02009b4/users/72644a4c8db443658b19af14756daaf0/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:30.965115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ed150-d8eb-4e21-bec0-203e318108c2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:30.965408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ed150-d8eb-4e21-bec0-203e318108c2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:30.965684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ed150-d8eb-4e21-bec0-203e318108c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:30.992308 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bf8ed150-d8eb-4e21-bec0-203e318108c2 None None] MFA Rules not processed for user `72644a4c8db443658b19af14756daaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:30.993501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bf8ed150-d8eb-4e21-bec0-203e318108c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:31.057347 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1354/2730] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:31.068124 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:31.069838 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:31.121958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:31.125984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0753b9c-64f4-4725-8749-68d6991a0432 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:31.131443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc4b12-9d30-45a3-a1c9-d1bdcd75fe3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:31.132251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc4b12-9d30-45a3-a1c9-d1bdcd75fe3a service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:31.132756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc4b12-9d30-45a3-a1c9-d1bdcd75fe3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:31.133159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc4b12-9d30-45a3-a1c9-d1bdcd75fe3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:31.141814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc4b12-9d30-45a3-a1c9-d1bdcd75fe3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:31.143814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc4b12-9d30-45a3-a1c9-d1bdcd75fe3a service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:31.152729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1377/2731] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:41.054342 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:41.148847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:44.458431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ed150-d8eb-4e21-bec0-203e318108c2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:44.528149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ed150-d8eb-4e21-bec0-203e318108c2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:44.532533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:44.533436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:44.533719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:44.533991 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] PATH_INFO: `/v3/users/72644a4c8db443658b19af14756daaf0` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:44.536666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72644a4c8db443658b19af14756daaf0)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:44.538601 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:44.587048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72644a4c8db443658b19af14756daaf0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:44.598452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72644a4c8db443658b19af14756daaf0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:44.611267 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72644a4c8db443658b19af14756daaf0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:44.611967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72644a4c8db443658b19af14756daaf0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:44.619824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72644a4c8db443658b19af14756daaf0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:44.626456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1355/2732] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:37:44 2022] DELETE /identity/v3/users/72644a4c8db443658b19af14756daaf0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:44.631490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592bc90-10bf-4f81-8ce9-3c8ec8543b19 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:44.631962 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592bc90-10bf-4f81-8ce9-3c8ec8543b19 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:44.632081 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592bc90-10bf-4f81-8ce9-3c8ec8543b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:44.660852 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d592bc90-10bf-4f81-8ce9-3c8ec8543b19 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:44.662706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d592bc90-10bf-4f81-8ce9-3c8ec8543b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:44.728918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1378/2733] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:44.738777 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:44.740487 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:44.839146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:44.850060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce6c50-56bc-442a-979e-432091e8dc16 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:44.854049 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c83190-7041-418e-a106-82107a773b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:44.854704 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c83190-7041-418e-a106-82107a773b36 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:44.855014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c83190-7041-418e-a106-82107a773b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:44.855379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c83190-7041-418e-a106-82107a773b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:44.871866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c83190-7041-418e-a106-82107a773b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:44.873340 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c83190-7041-418e-a106-82107a773b36 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:44.885643 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1356/2734] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:45.249377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592bc90-10bf-4f81-8ce9-3c8ec8543b19 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:45.252551 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d592bc90-10bf-4f81-8ce9-3c8ec8543b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:45.292729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592bc90-10bf-4f81-8ce9-3c8ec8543b19 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:45.299843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:45.301414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.303232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.303232 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] PATH_INFO: `/v3/projects/924cd779de6f4c608248eae1c02009b4` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.303494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924cd779de6f4c608248eae1c02009b4)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:45.305405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:45.364743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924cd779de6f4c608248eae1c02009b4', 'request_id': 'req-478ff67f-8b89-4a0d-b670-240672e47778'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:45.370984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924cd779de6f4c608248eae1c02009b4', 'request_id': 'req-478ff67f-8b89-4a0d-b670-240672e47778'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:45.376560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924cd779de6f4c608248eae1c02009b4', 'request_id': 'req-478ff67f-8b89-4a0d-b670-240672e47778'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:45.391092 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] The token cache is being invalidate because project 924cd779de6f4c608248eae1c02009b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:37:45.391552 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:45.392229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-478ff67f-8b89-4a0d-b670-240672e47778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:37:45.393871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1379/2735] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:37:45 2022] DELETE /identity/v3/projects/924cd779de6f4c608248eae1c02009b4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:45.399055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d462-6502-4eba-998c-fdf5cc139256 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.399542 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d462-6502-4eba-998c-fdf5cc139256 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.404656 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d462-6502-4eba-998c-fdf5cc139256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.449729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0bf5d462-6502-4eba-998c-fdf5cc139256 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:45.451033 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0bf5d462-6502-4eba-998c-fdf5cc139256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:45.514467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1357/2736] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:45.523101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6a76-9265-4758-be7b-784534fe7e11 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.530372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6a76-9265-4758-be7b-784534fe7e11 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.530372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6a76-9265-4758-be7b-784534fe7e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.544168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e78f6a76-9265-4758-be7b-784534fe7e11 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:45.545397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e78f6a76-9265-4758-be7b-784534fe7e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:45.556182 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1380/2737] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:45.560076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d462-6502-4eba-998c-fdf5cc139256 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:45.570989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d462-6502-4eba-998c-fdf5cc139256 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:45.575725 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:45.576394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.576686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.576998 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.577730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:37:45.578041 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:45.579666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:45.596880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1358/2738] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:37:45.601882 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f6a76-9265-4758-be7b-784534fe7e11 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:45.611117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f6a76-9265-4758-be7b-784534fe7e11 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:45.616663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b258d69-226c-44e4-8be8-32d16f4830b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:45.617321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b258d69-226c-44e4-8be8-32d16f4830b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.617608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b258d69-226c-44e4-8be8-32d16f4830b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.617884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b258d69-226c-44e4-8be8-32d16f4830b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.618577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b258d69-226c-44e4-8be8-32d16f4830b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:45.620517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b258d69-226c-44e4-8be8-32d16f4830b4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:45.647993 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1381/2739] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:37:45 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:37:45.653551 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:45.664728 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd796511-286d-4287-b009-0a89944cea69 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:45.734522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0faa77-a019-4150-bd89-d9709cc43a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:45.735636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0faa77-a019-4150-bd89-d9709cc43a8d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.736332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0faa77-a019-4150-bd89-d9709cc43a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.736332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0faa77-a019-4150-bd89-d9709cc43a8d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.737165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0faa77-a019-4150-bd89-d9709cc43a8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:45.739990 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0faa77-a019-4150-bd89-d9709cc43a8d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:45.773104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1359/2740] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:37:45 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:37:45.779751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b258d69-226c-44e4-8be8-32d16f4830b4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:45.798592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b258d69-226c-44e4-8be8-32d16f4830b4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:45.805221 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:45.805970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.806325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.806673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.807517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:45.809231 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:45.820563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1382/2741] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:45.824585 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0faa77-a019-4150-bd89-d9709cc43a8d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:45.836146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0faa77-a019-4150-bd89-d9709cc43a8d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:45.840021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:45.841772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a4d48-b211-4851-b7c2-569a3d1fa11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:45.842197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:45.842473 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a4d48-b211-4851-b7c2-569a3d1fa11e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.842801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a4d48-b211-4851-b7c2-569a3d1fa11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.843125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a4d48-b211-4851-b7c2-569a3d1fa11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.844096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a4d48-b211-4851-b7c2-569a3d1fa11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:45.845792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a4d48-b211-4851-b7c2-569a3d1fa11e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:45.860599 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1360/2742] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:37:45.861645 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a4d48-b211-4851-b7c2-569a3d1fa11e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:45.873014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a4d48-b211-4851-b7c2-569a3d1fa11e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:45.880480 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d63636-7e35-4840-8c89-7db3a7aacaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:45.881383 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63636-7e35-4840-8c89-7db3a7aacaca admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.881666 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63636-7e35-4840-8c89-7db3a7aacaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.882011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63636-7e35-4840-8c89-7db3a7aacaca admin admin] PATH_INFO: `/v3/projects/f785fe1f5aea4aaaaac50187f1ef765a/users/89f2c92f417f42e6957bc97a27782092/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.898691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:45.905332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d63636-7e35-4840-8c89-7db3a7aacaca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f785fe1f5aea4aaaaac50187f1ef765a, user_id=89f2c92f417f42e6957bc97a27782092, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:45.908328 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d63636-7e35-4840-8c89-7db3a7aacaca admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:45.922504 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1361/2743] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:37:45 2022] PUT /identity/v3/projects/f785fe1f5aea4aaaaac50187f1ef765a/users/89f2c92f417f42e6957bc97a27782092/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:37:45.932610 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1df0aa-029e-45ec-905f-e340f0cba46e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:45.932904 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1df0aa-029e-45ec-905f-e340f0cba46e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:45.933191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1df0aa-029e-45ec-905f-e340f0cba46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:45.939285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1ee8-ebd7-4678-a0e9-373b07a6b7d4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:45.959014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:45.961975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2a1df0aa-029e-45ec-905f-e340f0cba46e None None] MFA Rules not processed for user `89f2c92f417f42e6957bc97a27782092`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:45.964067 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2a1df0aa-029e-45ec-905f-e340f0cba46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:46.009443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c83f89-01bb-462a-882f-ae796078da68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:46.010100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c83f89-01bb-462a-882f-ae796078da68 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:46.010383 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c83f89-01bb-462a-882f-ae796078da68 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:46.010660 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c83f89-01bb-462a-882f-ae796078da68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:46.012506 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-67c83f89-01bb-462a-882f-ae796078da68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:46.025959 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c83f89-01bb-462a-882f-ae796078da68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:46.027507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c83f89-01bb-462a-882f-ae796078da68 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:46.032621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1362/2744] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:37:46.039321 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1383/2745] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:46.044012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1df0aa-029e-45ec-905f-e340f0cba46e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:46.045512 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2a1df0aa-029e-45ec-905f-e340f0cba46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:46.106924 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1df0aa-029e-45ec-905f-e340f0cba46e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:46.110587 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:46.111243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:46.111557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:46.111840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:46.120401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:46.121838 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:46.131148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1363/2746] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:47.493536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72cfa7-ab49-4270-a6aa-da41c5d9d156 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:47.494082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72cfa7-ab49-4270-a6aa-da41c5d9d156 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:47.494082 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72cfa7-ab49-4270-a6aa-da41c5d9d156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:47.510900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2c72cfa7-ab49-4270-a6aa-da41c5d9d156 None None] MFA Rules not processed for user `89f2c92f417f42e6957bc97a27782092`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:47.512120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2c72cfa7-ab49-4270-a6aa-da41c5d9d156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:47.519096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1384/2747] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:37:50.693927 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:37:50.696058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:50.764741 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120e659-2fd5-4d67-9de8-321f9e5f50b5 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:37:50.768903 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:37:50.769605 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:50.769889 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:50.770164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:50.772101 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:50.841072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:37:50.843709 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:37:50.845515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:37:50.857502 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1364/2748] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:37:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:37:54.392279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aa6a2-89f6-4ed4-b97e-e0c388011a21 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:37:54.392836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aa6a2-89f6-4ed4-b97e-e0c388011a21 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:37:54.392836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aa6a2-89f6-4ed4-b97e-e0c388011a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:37:54.412235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d85aa6a2-89f6-4ed4-b97e-e0c388011a21 None None] MFA Rules not processed for user `7343386004e24474bc0af9126f47182b`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:37:54.413436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d85aa6a2-89f6-4ed4-b97e-e0c388011a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:37:54.470837 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1385/2749] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:37:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:00.852778 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:04.464368 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=73 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:06.260963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:06.262401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:06.326077 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b80e7-12ef-4809-a6bb-735772fda7b9 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:06.330792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:06.331811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:06.332179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:06.332459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] PATH_INFO: `/v3/users/732e20214fbe402088ccaf513f664dee` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:06.333675 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=732e20214fbe402088ccaf513f664dee)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:06.335631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:06.382353 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '732e20214fbe402088ccaf513f664dee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:06.389249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '732e20214fbe402088ccaf513f664dee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:06.401009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '732e20214fbe402088ccaf513f664dee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:06.401578 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '732e20214fbe402088ccaf513f664dee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:06.408238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '732e20214fbe402088ccaf513f664dee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:06.414262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1365/2750] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:38:06 2022] DELETE /identity/v3/users/732e20214fbe402088ccaf513f664dee => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:38:06.418707 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d061-54c3-4417-b175-41f785801c91 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:06.419173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d061-54c3-4417-b175-41f785801c91 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:06.419299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d061-54c3-4417-b175-41f785801c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:06.437105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2506d061-54c3-4417-b175-41f785801c91 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:06.438281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2506d061-54c3-4417-b175-41f785801c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:06.498682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1386/2751] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:06.509319 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:06.510806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:06.575263 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157169d8-42df-4013-b460-114fb0594897 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:06.578893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:06.579592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:06.579874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:06.580146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:06.589829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:06.591564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:06.602024 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1366/2752] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:38:06.666814 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5ecb-235b-4ab7-bbe6-31ceeaf9e153 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:06.667104 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5ecb-235b-4ab7-bbe6-31ceeaf9e153 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:06.667415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5ecb-235b-4ab7-bbe6-31ceeaf9e153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:06.697494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-79fa5ecb-235b-4ab7-bbe6-31ceeaf9e153 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:06.698699 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-79fa5ecb-235b-4ab7-bbe6-31ceeaf9e153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:06.763185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1387/2753] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:06.811118 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:06.812561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:06.843137 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:06.891184 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa5ecb-235b-4ab7-bbe6-31ceeaf9e153 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:06.892824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-79fa5ecb-235b-4ab7-bbe6-31ceeaf9e153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:06.907765 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fd82b-f621-4e29-9f79-5608eda086c1 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:06.908816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa5ecb-235b-4ab7-bbe6-31ceeaf9e153 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:06.914447 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e1aa6-a530-4372-8c72-78b0d1ad4b78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:06.915144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e1aa6-a530-4372-8c72-78b0d1ad4b78 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:06.915538 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e1aa6-a530-4372-8c72-78b0d1ad4b78 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:06.915890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e1aa6-a530-4372-8c72-78b0d1ad4b78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:06.925441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:06.926558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:06.926956 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:06.927333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] PATH_INFO: `/v3/projects/0b0cf96d4c8b4e1e96ccde35364e85de` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:06.928932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b0cf96d4c8b4e1e96ccde35364e85de)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:06.930482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e1aa6-a530-4372-8c72-78b0d1ad4b78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:06.931457 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:06.932796 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e1aa6-a530-4372-8c72-78b0d1ad4b78 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:06.944055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1367/2754] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:38:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 17:38:06.992517 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b0cf96d4c8b4e1e96ccde35364e85de', 'request_id': 'req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:07.001358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b0cf96d4c8b4e1e96ccde35364e85de', 'request_id': 'req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:07.013338 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b0cf96d4c8b4e1e96ccde35364e85de', 'request_id': 'req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:07.028588 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] The token cache is being invalidate because project 0b0cf96d4c8b4e1e96ccde35364e85de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:38:07.031293 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:07.032385 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a1d36bfa-6b02-483f-a304-df7e5e0c64e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:07.038657 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1388/2755] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:38:06 2022] DELETE /identity/v3/projects/0b0cf96d4c8b4e1e96ccde35364e85de => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:38:07.040806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37bced-f35b-4c60-97ec-db6bd0520eb9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.040806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37bced-f35b-4c60-97ec-db6bd0520eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.041155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37bced-f35b-4c60-97ec-db6bd0520eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.074010 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5a37bced-f35b-4c60-97ec-db6bd0520eb9 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:07.075211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5a37bced-f35b-4c60-97ec-db6bd0520eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:07.151478 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1368/2756] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:07.159570 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a114c-5b5d-43ad-84c0-847a218bad79 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.159849 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a114c-5b5d-43ad-84c0-847a218bad79 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.160095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a114c-5b5d-43ad-84c0-847a218bad79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.180643 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a39a114c-5b5d-43ad-84c0-847a218bad79 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:07.181870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a39a114c-5b5d-43ad-84c0-847a218bad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:07.190397 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1389/2757] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:07.203035 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37bced-f35b-4c60-97ec-db6bd0520eb9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:07.214839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37bced-f35b-4c60-97ec-db6bd0520eb9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:07.220044 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:07.220761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.221048 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.221327 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.222404 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:38:07.223256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:07.224970 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:07.241217 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1369/2758] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:38:07.246518 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a114c-5b5d-43ad-84c0-847a218bad79 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:07.258276 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a114c-5b5d-43ad-84c0-847a218bad79 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:07.263746 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3250f-4bb9-43ce-89bc-e8526b65da2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:07.264405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3250f-4bb9-43ce-89bc-e8526b65da2b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.264689 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3250f-4bb9-43ce-89bc-e8526b65da2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.264967 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3250f-4bb9-43ce-89bc-e8526b65da2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.267549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3250f-4bb9-43ce-89bc-e8526b65da2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:07.267812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3250f-4bb9-43ce-89bc-e8526b65da2b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:07.279174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:07.303399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1390/2759] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:38:07 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:38:07.306536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:07.317577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a107161-b703-4399-b55b-ec2fb25977a5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:07.391141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a31a65-80b2-4b15-96c5-58f28e684e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:07.391862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a31a65-80b2-4b15-96c5-58f28e684e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.392177 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a31a65-80b2-4b15-96c5-58f28e684e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.392459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a31a65-80b2-4b15-96c5-58f28e684e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.393165 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a31a65-80b2-4b15-96c5-58f28e684e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:07.395072 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a31a65-80b2-4b15-96c5-58f28e684e0d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:07.419960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1370/2760] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:38:07 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:38:07.421147 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3250f-4bb9-43ce-89bc-e8526b65da2b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:07.435228 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3250f-4bb9-43ce-89bc-e8526b65da2b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:07.440546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2af1b4-8360-4312-83f6-8ae3116e2f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:07.441722 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2af1b4-8360-4312-83f6-8ae3116e2f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.442069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2af1b4-8360-4312-83f6-8ae3116e2f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.442391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2af1b4-8360-4312-83f6-8ae3116e2f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.443112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2af1b4-8360-4312-83f6-8ae3116e2f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:07.444794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2af1b4-8360-4312-83f6-8ae3116e2f22 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:07.458007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1391/2761] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:38:07.461853 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a31a65-80b2-4b15-96c5-58f28e684e0d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:07.484163 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a31a65-80b2-4b15-96c5-58f28e684e0d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:07.491150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769dc6c-c833-4638-a7c1-18e151bab655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:07.491910 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769dc6c-c833-4638-a7c1-18e151bab655 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.492191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769dc6c-c833-4638-a7c1-18e151bab655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.492448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769dc6c-c833-4638-a7c1-18e151bab655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.493191 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769dc6c-c833-4638-a7c1-18e151bab655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:07.494795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769dc6c-c833-4638-a7c1-18e151bab655 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:07.506499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1371/2762] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:38:07.510792 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2af1b4-8360-4312-83f6-8ae3116e2f22 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:07.522626 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2af1b4-8360-4312-83f6-8ae3116e2f22 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:07.527979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a000bb-3147-4109-b29b-9ff246b55f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:07.530683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a000bb-3147-4109-b29b-9ff246b55f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.534116 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a000bb-3147-4109-b29b-9ff246b55f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.535072 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a000bb-3147-4109-b29b-9ff246b55f9b admin admin] PATH_INFO: `/v3/projects/748923c17bfd44459ba985685d1e642d/users/db5da087b4b5444ca6a11770103f5187/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.558904 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a000bb-3147-4109-b29b-9ff246b55f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=748923c17bfd44459ba985685d1e642d, user_id=db5da087b4b5444ca6a11770103f5187, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:07.562077 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a000bb-3147-4109-b29b-9ff246b55f9b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:07.573385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d7db-7ab1-4159-a490-c6e2a3fdeb1e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.573679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d7db-7ab1-4159-a490-c6e2a3fdeb1e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.574074 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d7db-7ab1-4159-a490-c6e2a3fdeb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.580095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1392/2763] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:38:07 2022] PUT /identity/v3/projects/748923c17bfd44459ba985685d1e642d/users/db5da087b4b5444ca6a11770103f5187/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:38:07.590595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6399e-e378-49f0-96e7-b312382083a4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.590935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6399e-e378-49f0-96e7-b312382083a4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.591452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6399e-e378-49f0-96e7-b312382083a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.595960 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9949d7db-7ab1-4159-a490-c6e2a3fdeb1e None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:07.597867 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9949d7db-7ab1-4159-a490-c6e2a3fdeb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:07.603683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:07.629733 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-64a6399e-e378-49f0-96e7-b312382083a4 None None] MFA Rules not processed for user `db5da087b4b5444ca6a11770103f5187`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:07.630992 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-64a6399e-e378-49f0-96e7-b312382083a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:07.662951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1372/2764] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 17:38:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:07.676017 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949d7db-7ab1-4159-a490-c6e2a3fdeb1e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:07.677641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9949d7db-7ab1-4159-a490-c6e2a3fdeb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:07.703649 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1393/2765] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:38:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:38:07.716450 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6399e-e378-49f0-96e7-b312382083a4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:07.718101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-64a6399e-e378-49f0-96e7-b312382083a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:07.758560 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949d7db-7ab1-4159-a490-c6e2a3fdeb1e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:07.765747 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb01b74-9cae-4b2b-8f81-5c7c59394dd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:07.766408 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01b74-9cae-4b2b-8f81-5c7c59394dd0 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.766686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01b74-9cae-4b2b-8f81-5c7c59394dd0 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.767084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01b74-9cae-4b2b-8f81-5c7c59394dd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.781631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb01b74-9cae-4b2b-8f81-5c7c59394dd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:07.783503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6399e-e378-49f0-96e7-b312382083a4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:07.785352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb01b74-9cae-4b2b-8f81-5c7c59394dd0 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:07.787912 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d2157-57ed-4650-b546-5d7be7ccb6b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:07.788572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d2157-57ed-4650-b546-5d7be7ccb6b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:07.788855 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d2157-57ed-4650-b546-5d7be7ccb6b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:07.789129 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d2157-57ed-4650-b546-5d7be7ccb6b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:07.794865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1373/2766] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:38:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 17:38:07.799738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d2157-57ed-4650-b546-5d7be7ccb6b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:07.801377 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d2157-57ed-4650-b546-5d7be7ccb6b8 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:07.811717 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1394/2767] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:38:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:38:08.000951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2016afe-038d-4fa3-a9e4-748abea531b4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:08.001508 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2016afe-038d-4fa3-a9e4-748abea531b4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:08.002073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2016afe-038d-4fa3-a9e4-748abea531b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:08.022863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a2016afe-038d-4fa3-a9e4-748abea531b4 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:08.024211 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a2016afe-038d-4fa3-a9e4-748abea531b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:08.034016 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1374/2768] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 17:38:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:38:08.047816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d2157-57ed-4650-b546-5d7be7ccb6b8 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:08.077226 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d2157-57ed-4650-b546-5d7be7ccb6b8 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:08.084816 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36812180-4030-4b83-8385-675e028e4263 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:08.085705 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36812180-4030-4b83-8385-675e028e4263 service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:08.086384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36812180-4030-4b83-8385-675e028e4263 service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:08.086954 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36812180-4030-4b83-8385-675e028e4263 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:08.111038 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36812180-4030-4b83-8385-675e028e4263 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:08.122491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36812180-4030-4b83-8385-675e028e4263 service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:08.151798 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1395/2769] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:38:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:38:08.479354 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d803fc-b712-4a98-8b6e-ea8b1b5f95c7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:08.479829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d803fc-b712-4a98-8b6e-ea8b1b5f95c7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:08.479950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d803fc-b712-4a98-8b6e-ea8b1b5f95c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:08.497696 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-10d803fc-b712-4a98-8b6e-ea8b1b5f95c7 None None] MFA Rules not processed for user `ba50db32ce9f444e919e667305594d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:08.498905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-10d803fc-b712-4a98-8b6e-ea8b1b5f95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:08.506655 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1375/2770] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 17:38:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:08.518516 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36812180-4030-4b83-8385-675e028e4263 service swift] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:08.529466 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36812180-4030-4b83-8385-675e028e4263 service swift] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:08.534543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0b5ef-3c2d-4028-9d6a-e95502d254dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4224f58ecbb423dae16d5b9ae09d5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:08.535379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b5ef-3c2d-4028-9d6a-e95502d254dd service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:08.535669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b5ef-3c2d-4028-9d6a-e95502d254dd service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:08.535950 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b5ef-3c2d-4028-9d6a-e95502d254dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:08.544154 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0b5ef-3c2d-4028-9d6a-e95502d254dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:08.545737 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0b5ef-3c2d-4028-9d6a-e95502d254dd service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:08.554671 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1396/2771] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 17:38:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 17:38:13.829289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d803fc-b712-4a98-8b6e-ea8b1b5f95c7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:13.830784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-10d803fc-b712-4a98-8b6e-ea8b1b5f95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:13.897381 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d803fc-b712-4a98-8b6e-ea8b1b5f95c7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:13.901830 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:13.902695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:13.902976 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:13.903249 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] PATH_INFO: `/v3/users/89f2c92f417f42e6957bc97a27782092` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:13.904471 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f2c92f417f42e6957bc97a27782092)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:13.906369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:13.941776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:13.954673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f2c92f417f42e6957bc97a27782092'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:13.961841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f2c92f417f42e6957bc97a27782092'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:13.974046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f2c92f417f42e6957bc97a27782092'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:13.974781 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f2c92f417f42e6957bc97a27782092'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:13.981598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f2c92f417f42e6957bc97a27782092'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:13.988096 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1376/2772] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:38:13 2022] DELETE /identity/v3/users/89f2c92f417f42e6957bc97a27782092 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:38:13.992547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26ffa-be9c-43e5-8fc6-4ccffef5e201 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:13.993009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26ffa-be9c-43e5-8fc6-4ccffef5e201 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:13.993133 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26ffa-be9c-43e5-8fc6-4ccffef5e201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:14.010963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dba26ffa-be9c-43e5-8fc6-4ccffef5e201 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:14.012240 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dba26ffa-be9c-43e5-8fc6-4ccffef5e201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:14.028193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:14.070443 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1397/2773] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:14.079858 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:14.081251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:14.138566 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e36db-7fa0-4346-bd42-690f5cbdbc22 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:14.142162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:14.142803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:14.143085 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:14.143391 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:14.152399 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:14.153816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:14.163788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1377/2774] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:38:14.332295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba26ffa-be9c-43e5-8fc6-4ccffef5e201 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:14.333791 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dba26ffa-be9c-43e5-8fc6-4ccffef5e201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:14.346030 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba26ffa-be9c-43e5-8fc6-4ccffef5e201 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:14.351522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:14.352833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:14.352833 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:14.353103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] PATH_INFO: `/v3/projects/f785fe1f5aea4aaaaac50187f1ef765a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:14.354100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f785fe1f5aea4aaaaac50187f1ef765a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:14.356035 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:14.403405 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f785fe1f5aea4aaaaac50187f1ef765a', 'request_id': 'req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:14.410251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f785fe1f5aea4aaaaac50187f1ef765a', 'request_id': 'req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:14.415790 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f785fe1f5aea4aaaaac50187f1ef765a', 'request_id': 'req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:14.428664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] The token cache is being invalidate because project f785fe1f5aea4aaaaac50187f1ef765a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:38:14.428949 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:14.429575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-eb06ffb0-7a4c-4aab-b58a-8732ab4a17a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:14.431214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1398/2775] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:38:14 2022] DELETE /identity/v3/projects/f785fe1f5aea4aaaaac50187f1ef765a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:38:24.158888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=43,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:24.426357 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:30.382982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:30.384419 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:30.397460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcee4f-9004-41d9-983b-9dac9e6f13f2 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:30.452795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:30.453673 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:30.453950 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:30.454220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] PATH_INFO: `/v3/users/db5da087b4b5444ca6a11770103f5187` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:30.455458 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5da087b4b5444ca6a11770103f5187)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:30.457934 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:30.501806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5da087b4b5444ca6a11770103f5187'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:30.509814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5da087b4b5444ca6a11770103f5187'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:30.522225 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5da087b4b5444ca6a11770103f5187'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:30.522891 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5da087b4b5444ca6a11770103f5187'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:30.529622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5da087b4b5444ca6a11770103f5187'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:30.535352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1378/2776] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:38:30 2022] DELETE /identity/v3/users/db5da087b4b5444ca6a11770103f5187 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:38:30.539924 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600a544-52bf-4873-9eda-e855a07c0b1d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:30.540429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600a544-52bf-4873-9eda-e855a07c0b1d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:30.540539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600a544-52bf-4873-9eda-e855a07c0b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:30.557764 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4600a544-52bf-4873-9eda-e855a07c0b1d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:38:30.558954 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4600a544-52bf-4873-9eda-e855a07c0b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:30.617235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1399/2777] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:38:30.626764 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:30.628424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:30.684590 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea21434-2fa9-4451-9655-0fb94e90eef1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:30.688332 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:30.688995 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:30.689273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:30.689546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:30.698896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:30.700331 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:30.710703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1379/2778] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:38:30.972851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600a544-52bf-4873-9eda-e855a07c0b1d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:38:30.974958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4600a544-52bf-4873-9eda-e855a07c0b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:38:30.986602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600a544-52bf-4873-9eda-e855a07c0b1d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:38:30.992482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:38:30.994510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:38:30.994813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:38:30.995090 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] PATH_INFO: `/v3/projects/748923c17bfd44459ba985685d1e642d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:38:30.996348 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=748923c17bfd44459ba985685d1e642d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:38:30.998832 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:38:31.048376 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '748923c17bfd44459ba985685d1e642d', 'request_id': 'req-277a356d-ba98-479d-b47e-447869a35aa9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:31.055115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '748923c17bfd44459ba985685d1e642d', 'request_id': 'req-277a356d-ba98-479d-b47e-447869a35aa9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:31.060861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '748923c17bfd44459ba985685d1e642d', 'request_id': 'req-277a356d-ba98-479d-b47e-447869a35aa9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:31.072894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] The token cache is being invalidate because project 748923c17bfd44459ba985685d1e642d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:38:31.073177 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:31.073774 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:38:31.075278 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1400/2779] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:38:30 2022] DELETE /identity/v3/projects/748923c17bfd44459ba985685d1e642d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:38:40.705339 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:38:41.070059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:39:24.616758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:39:24.618788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:39:24.691092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c78c-b651-41ba-b4d3-150dde787b55 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:39:24.752256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b09506-0dfd-41cc-a90b-269b8e21cebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:39:24.752905 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b09506-0dfd-41cc-a90b-269b8e21cebf service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:39:24.753185 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b09506-0dfd-41cc-a90b-269b8e21cebf service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:39:24.753456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b09506-0dfd-41cc-a90b-269b8e21cebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:39:24.755757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-80b09506-0dfd-41cc-a90b-269b8e21cebf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:39:24.811544 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b09506-0dfd-41cc-a90b-269b8e21cebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:39:24.813046 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b09506-0dfd-41cc-a90b-269b8e21cebf service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:39:24.822194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1380/2780] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:39:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:39:34.817897 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:39:38.549934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:39:38.551410 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:39:38.653218 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a356d-ba98-479d-b47e-447869a35aa9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:39:38.659033 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:39:38.659716 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:39:38.660031 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:39:38.660309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:39:38.662787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:39:38.674794 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:39:38.676366 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:39:38.686851 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1401/2781] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:39:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:39:42.041012 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c9d5-b423-4b5d-af7f-335a0a8bb478 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:39:42.041556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c9d5-b423-4b5d-af7f-335a0a8bb478 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:39:42.041556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c9d5-b423-4b5d-af7f-335a0a8bb478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:39:42.059515 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8924c9d5-b423-4b5d-af7f-335a0a8bb478 None None] MFA Rules not processed for user `7343386004e24474bc0af9126f47182b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:39:42.060767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8924c9d5-b423-4b5d-af7f-335a0a8bb478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:39:42.070344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1381/2782] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:39:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:39:42.367308 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:39:42.368767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:39:42.426420 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2f457-2216-4a6f-a54a-5429a17380eb service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:39:42.430280 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:39:42.430972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:39:42.431200 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:39:42.431504 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:39:42.433238 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:39:42.442785 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:39:42.444333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:39:42.452197 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1402/2783] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:39:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:39:52.054057 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:39:52.448639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=67 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:40:24.537158 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250d9d9-e27a-44ca-8836-f76fd9a172d5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:40:24.537641 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250d9d9-e27a-44ca-8836-f76fd9a172d5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:40:24.537762 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250d9d9-e27a-44ca-8836-f76fd9a172d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:40:24.555316 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e250d9d9-e27a-44ca-8836-f76fd9a172d5 None None] MFA Rules not processed for user `4a827bedc4b84508bb7770a09efee3f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:40:24.556536 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e250d9d9-e27a-44ca-8836-f76fd9a172d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:40:24.613711 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1382/2784] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:40:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:40:28.627381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:40:28.639970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235356ac-4792-44e8-8b63-bdc0a60bad74 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:40:28.644864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:40:28.645940 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:40:28.646430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:40:28.646929 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:40:28.649520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:40:28.689894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:40:28.691756 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:40:28.714250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1403/2785] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:40:29.930559 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250d9d9-e27a-44ca-8836-f76fd9a172d5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:40:29.932000 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e250d9d9-e27a-44ca-8836-f76fd9a172d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:40:29.943723 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250d9d9-e27a-44ca-8836-f76fd9a172d5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:40:29.947535 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:40:29.948212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:40:29.948493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:40:29.948767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:40:29.950634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:40:29.964484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:40:29.965909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:40:29.977834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1383/2786] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:40:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:40:38.711425 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=18 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:40:39.972155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=31 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:30.620677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:30.630235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a85a6a-d26e-4988-a961-8db52b7133fe service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:30.633830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:30.634468 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:30.634750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:30.635026 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:30.636789 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:30.699176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:30.701267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:30.710695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1404/2787] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:33.486023 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:33.487630 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:33.512611 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9f981-8284-4f75-a057-0a73ccd48fd5 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:33.519597 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:33.520543 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:33.520834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:33.521120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:33.523036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:33.536155 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:33.537964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:33.550122 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1384/2788] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:33.563875 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:33.565414 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:33.584239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd718e1-241c-4021-a3f9-34caebf994ac service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:33.589337 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:33.589847 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:33.590130 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:33.590406 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] PATH_INFO: `/v3/users/7343386004e24474bc0af9126f47182b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:33.591740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7343386004e24474bc0af9126f47182b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:33.594295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:33.649372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7343386004e24474bc0af9126f47182b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:33.650019 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7343386004e24474bc0af9126f47182b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:33.656897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7343386004e24474bc0af9126f47182b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:33.668930 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7343386004e24474bc0af9126f47182b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:33.669666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-307a22c1-84ad-47cc-8f55-f9b3f7cbaf22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7343386004e24474bc0af9126f47182b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:33.682224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1405/2789] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:41:33 2022] DELETE /identity/v3/users/7343386004e24474bc0af9126f47182b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:33.856227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:33.869491 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4ec4-519f-401e-9204-69ceafe4cfc1 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:33.927215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:33.928109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:33.928386 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:33.928651 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] PATH_INFO: `/v3/projects/9525b35792a04dc3a6f109cb39ee9d50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:33.929819 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9525b35792a04dc3a6f109cb39ee9d50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:33.931750 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:33.979526 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9525b35792a04dc3a6f109cb39ee9d50', 'request_id': 'req-a57de6c2-5049-4f76-83f4-2e1a47008b11'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:33.987499 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9525b35792a04dc3a6f109cb39ee9d50', 'request_id': 'req-a57de6c2-5049-4f76-83f4-2e1a47008b11'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:33.993975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9525b35792a04dc3a6f109cb39ee9d50', 'request_id': 'req-a57de6c2-5049-4f76-83f4-2e1a47008b11'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:34.005148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] The token cache is being invalidate because project 9525b35792a04dc3a6f109cb39ee9d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:41:34.005433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:34.006075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a57de6c2-5049-4f76-83f4-2e1a47008b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:34.007682 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1385/2790] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:41:33 2022] DELETE /identity/v3/projects/9525b35792a04dc3a6f109cb39ee9d50 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:34.013194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ac710-8c19-467d-83be-e67295113b90 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.013491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ac710-8c19-467d-83be-e67295113b90 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.013871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ac710-8c19-467d-83be-e67295113b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.031549 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-291ac710-8c19-467d-83be-e67295113b90 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:34.032743 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-291ac710-8c19-467d-83be-e67295113b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:34.092885 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1406/2791] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:34.100788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b427-0ee5-493c-bf4c-f649bad67f06 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.101080 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b427-0ee5-493c-bf4c-f649bad67f06 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.101359 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b427-0ee5-493c-bf4c-f649bad67f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.120951 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d2e9b427-0ee5-493c-bf4c-f649bad67f06 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:34.122246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d2e9b427-0ee5-493c-bf4c-f649bad67f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:34.130400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1386/2792] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:34.134659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ac710-8c19-467d-83be-e67295113b90 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.144727 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ac710-8c19-467d-83be-e67295113b90 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.149268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.149917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.150201 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.150476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.151205 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:41:34.151553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.153159 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.155202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:34.167327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1407/2793] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:41:34.172190 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9b427-0ee5-493c-bf4c-f649bad67f06 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.181369 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9b427-0ee5-493c-bf4c-f649bad67f06 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.186401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc552f7-c38b-4db8-8f34-34af1676556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.187066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc552f7-c38b-4db8-8f34-34af1676556b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.187412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc552f7-c38b-4db8-8f34-34af1676556b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.187695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc552f7-c38b-4db8-8f34-34af1676556b admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.188432 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc552f7-c38b-4db8-8f34-34af1676556b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.190393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc552f7-c38b-4db8-8f34-34af1676556b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.218562 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1387/2794] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:41:34 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:34.222721 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.232781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154704d-88ec-4753-9a22-a42e3336a5c1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.290836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32450d87-f0d0-48dd-93d3-ddf37624f089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.291538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32450d87-f0d0-48dd-93d3-ddf37624f089 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.291825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32450d87-f0d0-48dd-93d3-ddf37624f089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.292105 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32450d87-f0d0-48dd-93d3-ddf37624f089 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.292843 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32450d87-f0d0-48dd-93d3-ddf37624f089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.295111 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32450d87-f0d0-48dd-93d3-ddf37624f089 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.315636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1408/2795] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:41:34 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:34.319620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc552f7-c38b-4db8-8f34-34af1676556b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.330415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc552f7-c38b-4db8-8f34-34af1676556b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.334981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cff6ca-647b-4985-b036-b8ff74596e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.335671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff6ca-647b-4985-b036-b8ff74596e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.335959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff6ca-647b-4985-b036-b8ff74596e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.336236 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff6ca-647b-4985-b036-b8ff74596e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.336965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cff6ca-647b-4985-b036-b8ff74596e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.338556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cff6ca-647b-4985-b036-b8ff74596e8f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.348902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1388/2796] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:34.352490 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32450d87-f0d0-48dd-93d3-ddf37624f089 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.362595 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32450d87-f0d0-48dd-93d3-ddf37624f089 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.367216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89134cd-7d70-4443-bfe3-f1f1ffc302af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.367900 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89134cd-7d70-4443-bfe3-f1f1ffc302af admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.368185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89134cd-7d70-4443-bfe3-f1f1ffc302af admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.368460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89134cd-7d70-4443-bfe3-f1f1ffc302af admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.369189 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89134cd-7d70-4443-bfe3-f1f1ffc302af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.370899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89134cd-7d70-4443-bfe3-f1f1ffc302af admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.381796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1409/2797] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:34.385772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cff6ca-647b-4985-b036-b8ff74596e8f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.396376 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cff6ca-647b-4985-b036-b8ff74596e8f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.400982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4481-7427-4667-819e-a319d1c76791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.401863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4481-7427-4667-819e-a319d1c76791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.402147 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4481-7427-4667-819e-a319d1c76791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.402422 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4481-7427-4667-819e-a319d1c76791 admin admin] PATH_INFO: `/v3/projects/d3759c76bc124a3297ff95c9d679811f/users/11e10eed4f6a48adadad73c0d8a3c8c9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.416916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c4481-7427-4667-819e-a319d1c76791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3759c76bc124a3297ff95c9d679811f, user_id=11e10eed4f6a48adadad73c0d8a3c8c9, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.419633 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c4481-7427-4667-819e-a319d1c76791 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.431955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1389/2798] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:34 2022] PUT /identity/v3/projects/d3759c76bc124a3297ff95c9d679811f/users/11e10eed4f6a48adadad73c0d8a3c8c9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:41:34.442526 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bb989-bd03-4b29-9934-dc5309918cd7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.442818 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bb989-bd03-4b29-9934-dc5309918cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.443093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bb989-bd03-4b29-9934-dc5309918cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.469354 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a09bb989-bd03-4b29-9934-dc5309918cd7 None None] MFA Rules not processed for user `11e10eed4f6a48adadad73c0d8a3c8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:34.470555 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a09bb989-bd03-4b29-9934-dc5309918cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:34.539399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1410/2799] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:34.543923 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4481-7427-4667-819e-a319d1c76791 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.557352 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:34.607844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4481-7427-4667-819e-a319d1c76791 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.612517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8085f-0dd8-4b00-bae7-9869dd54210f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.613180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8085f-0dd8-4b00-bae7-9869dd54210f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.613466 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8085f-0dd8-4b00-bae7-9869dd54210f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.613745 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8085f-0dd8-4b00-bae7-9869dd54210f admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.614444 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8085f-0dd8-4b00-bae7-9869dd54210f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.616388 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8085f-0dd8-4b00-bae7-9869dd54210f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.642294 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1390/2800] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:41:34 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:41:34.646358 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bb989-bd03-4b29-9934-dc5309918cd7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.657034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bb989-bd03-4b29-9934-dc5309918cd7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.712903 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144eec67-2e4d-4563-9dfb-1265f0777fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.713575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144eec67-2e4d-4563-9dfb-1265f0777fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.713853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144eec67-2e4d-4563-9dfb-1265f0777fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.714127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144eec67-2e4d-4563-9dfb-1265f0777fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.714819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144eec67-2e4d-4563-9dfb-1265f0777fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.717667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144eec67-2e4d-4563-9dfb-1265f0777fa2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.725384 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:34.739103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1411/2801] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:41:34 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:34.742460 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8085f-0dd8-4b00-bae7-9869dd54210f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.752981 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8085f-0dd8-4b00-bae7-9869dd54210f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.757506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa605f58-19bc-484d-bf85-a77b0c934168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.758156 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa605f58-19bc-484d-bf85-a77b0c934168 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.758436 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa605f58-19bc-484d-bf85-a77b0c934168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.758707 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa605f58-19bc-484d-bf85-a77b0c934168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.759462 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa605f58-19bc-484d-bf85-a77b0c934168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.761038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa605f58-19bc-484d-bf85-a77b0c934168 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.772431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1391/2802] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:34.776379 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144eec67-2e4d-4563-9dfb-1265f0777fa2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.787074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144eec67-2e4d-4563-9dfb-1265f0777fa2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.791724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade913b4-23c1-40f4-a81d-bd1724bb0a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.792592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade913b4-23c1-40f4-a81d-bd1724bb0a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.792869 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade913b4-23c1-40f4-a81d-bd1724bb0a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.793142 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade913b4-23c1-40f4-a81d-bd1724bb0a11 admin admin] PATH_INFO: `/v3/projects/83c238b7deb4427b92d7d225a04b2739/users/0a1fba66dd444e92b688c2d9742fbe67/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.807585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade913b4-23c1-40f4-a81d-bd1724bb0a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c238b7deb4427b92d7d225a04b2739, user_id=0a1fba66dd444e92b688c2d9742fbe67, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.810273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade913b4-23c1-40f4-a81d-bd1724bb0a11 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.822421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1412/2803] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:34 2022] PUT /identity/v3/projects/83c238b7deb4427b92d7d225a04b2739/users/0a1fba66dd444e92b688c2d9742fbe67/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:34.827144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa605f58-19bc-484d-bf85-a77b0c934168 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.836860 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa605f58-19bc-484d-bf85-a77b0c934168 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.892865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f00680-bae2-407e-8e70-b10c04e0354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.893459 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f00680-bae2-407e-8e70-b10c04e0354e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.893737 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f00680-bae2-407e-8e70-b10c04e0354e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.894009 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f00680-bae2-407e-8e70-b10c04e0354e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.894719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f00680-bae2-407e-8e70-b10c04e0354e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.896323 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f00680-bae2-407e-8e70-b10c04e0354e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.906517 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1392/2804] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:34.910618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade913b4-23c1-40f4-a81d-bd1724bb0a11 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:34.920772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade913b4-23c1-40f4-a81d-bd1724bb0a11 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:34.925269 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:34.926138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.926413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.926683 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] PATH_INFO: `/v3/projects/83c238b7deb4427b92d7d225a04b2739/users/0a1fba66dd444e92b688c2d9742fbe67/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.928644 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c238b7deb4427b92d7d225a04b2739, user_id=0a1fba66dd444e92b688c2d9742fbe67, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:34.931251 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:34.941399 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1413/2805] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:34 2022] PUT /identity/v3/projects/83c238b7deb4427b92d7d225a04b2739/users/0a1fba66dd444e92b688c2d9742fbe67/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:34.952893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd7c08-bfbe-482a-8876-62aacaa0154f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:34.953221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd7c08-bfbe-482a-8876-62aacaa0154f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:34.953531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd7c08-bfbe-482a-8876-62aacaa0154f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:34.979789 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-42bd7c08-bfbe-482a-8876-62aacaa0154f None None] MFA Rules not processed for user `0a1fba66dd444e92b688c2d9742fbe67`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:34.980989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42bd7c08-bfbe-482a-8876-62aacaa0154f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:35.052899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1393/2806] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:35.064674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:35.066395 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:35.158429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771396b0-801d-4fd2-bbdc-8a07cb7aefb6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:35.162168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:35.162842 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:35.163126 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:35.163429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:35.172553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:35.174117 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:35.185632 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1414/2807] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:35.259742 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd7c08-bfbe-482a-8876-62aacaa0154f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:35.269864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd7c08-bfbe-482a-8876-62aacaa0154f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:35.273510 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0bc20-ac82-428d-a8cb-0025ccffc3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:35.274141 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0bc20-ac82-428d-a8cb-0025ccffc3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:35.274417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0bc20-ac82-428d-a8cb-0025ccffc3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:35.274684 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0bc20-ac82-428d-a8cb-0025ccffc3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:35.280273 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:35.333429 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0bc20-ac82-428d-a8cb-0025ccffc3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:35.334863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0bc20-ac82-428d-a8cb-0025ccffc3a6 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:35.344051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1394/2808] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:38.429661 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:38.431067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:38.487972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbd240-a41d-4a04-a170-3606e42fef89 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:38.491566 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a379bf-ce93-45cb-96a6-f8ceaa6d57f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:38.492191 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a379bf-ce93-45cb-96a6-f8ceaa6d57f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:38.492475 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a379bf-ce93-45cb-96a6-f8ceaa6d57f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:38.492751 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a379bf-ce93-45cb-96a6-f8ceaa6d57f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:38.494501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b4a379bf-ce93-45cb-96a6-f8ceaa6d57f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:38.504076 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a379bf-ce93-45cb-96a6-f8ceaa6d57f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:38.505538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a379bf-ce93-45cb-96a6-f8ceaa6d57f3 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:38.513972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1415/2809] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:45.169481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0bc20-ac82-428d-a8cb-0025ccffc3a6 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:45.230695 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0bc20-ac82-428d-a8cb-0025ccffc3a6 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:45.235192 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:45.236090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:45.236368 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:45.236640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] PATH_INFO: `/v3/users/11e10eed4f6a48adadad73c0d8a3c8c9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:45.237816 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e10eed4f6a48adadad73c0d8a3c8c9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:45.239772 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:45.285303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e10eed4f6a48adadad73c0d8a3c8c9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.291568 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e10eed4f6a48adadad73c0d8a3c8c9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.303450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e10eed4f6a48adadad73c0d8a3c8c9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.304073 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e10eed4f6a48adadad73c0d8a3c8c9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.311325 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e10eed4f6a48adadad73c0d8a3c8c9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.317231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1395/2810] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:41:45 2022] DELETE /identity/v3/users/11e10eed4f6a48adadad73c0d8a3c8c9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:45.321678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8adf3a-09d2-4d42-b00e-cf0bdd28e51d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:45.322163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8adf3a-09d2-4d42-b00e-cf0bdd28e51d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:45.322268 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8adf3a-09d2-4d42-b00e-cf0bdd28e51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:45.339143 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cd8adf3a-09d2-4d42-b00e-cf0bdd28e51d None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:45.340347 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cd8adf3a-09d2-4d42-b00e-cf0bdd28e51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:45.375188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:45.402175 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1416/2811] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:45.412431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:45.413861 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:45.445030 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:45.473592 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28521c7-8113-48f7-9bd8-ba734a92358b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:45.477250 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86f26c-f30f-4940-adb2-52ad18cc8a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:45.477899 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86f26c-f30f-4940-adb2-52ad18cc8a73 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:45.478183 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86f26c-f30f-4940-adb2-52ad18cc8a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:45.478468 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86f26c-f30f-4940-adb2-52ad18cc8a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:45.487670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86f26c-f30f-4940-adb2-52ad18cc8a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:45.489103 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86f26c-f30f-4940-adb2-52ad18cc8a73 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:45.499167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1396/2812] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:45.757115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8adf3a-09d2-4d42-b00e-cf0bdd28e51d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:45.758576 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cd8adf3a-09d2-4d42-b00e-cf0bdd28e51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:45.771979 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8adf3a-09d2-4d42-b00e-cf0bdd28e51d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:45.776801 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:45.778578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:45.780039 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:45.780327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] PATH_INFO: `/v3/projects/d3759c76bc124a3297ff95c9d679811f` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:45.781599 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3759c76bc124a3297ff95c9d679811f)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:45.783577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:45.831005 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3759c76bc124a3297ff95c9d679811f', 'request_id': 'req-5ec6ac34-1790-4589-9f46-c0b2c91d811f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.837150 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3759c76bc124a3297ff95c9d679811f', 'request_id': 'req-5ec6ac34-1790-4589-9f46-c0b2c91d811f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.843881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3759c76bc124a3297ff95c9d679811f', 'request_id': 'req-5ec6ac34-1790-4589-9f46-c0b2c91d811f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.856048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] The token cache is being invalidate because project d3759c76bc124a3297ff95c9d679811f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:41:45.856333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.857193 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.858895 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1417/2813] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:41:45 2022] DELETE /identity/v3/projects/d3759c76bc124a3297ff95c9d679811f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:45.862758 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86f26c-f30f-4940-adb2-52ad18cc8a73 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:45.872276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86f26c-f30f-4940-adb2-52ad18cc8a73 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:45.930264 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:45.931148 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:45.931493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:45.931768 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] PATH_INFO: `/v3/users/0a1fba66dd444e92b688c2d9742fbe67` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:45.932956 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1fba66dd444e92b688c2d9742fbe67)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:45.934887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:45.981153 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1fba66dd444e92b688c2d9742fbe67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:45.988285 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1fba66dd444e92b688c2d9742fbe67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.000212 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1fba66dd444e92b688c2d9742fbe67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.000834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1fba66dd444e92b688c2d9742fbe67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.008018 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-61af4f65-54f0-494a-9a07-22b681739493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1fba66dd444e92b688c2d9742fbe67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.014667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1397/2814] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:41:45 2022] DELETE /identity/v3/users/0a1fba66dd444e92b688c2d9742fbe67 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:46.280066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:46.281677 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:46.298769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6ac34-1790-4589-9f46-c0b2c91d811f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:46.355575 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:46.356433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.356711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.357429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] PATH_INFO: `/v3/projects/83c238b7deb4427b92d7d225a04b2739` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.358543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c238b7deb4427b92d7d225a04b2739)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:46.360432 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:46.407750 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c238b7deb4427b92d7d225a04b2739', 'request_id': 'req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.413899 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c238b7deb4427b92d7d225a04b2739', 'request_id': 'req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.419439 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c238b7deb4427b92d7d225a04b2739', 'request_id': 'req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.422100 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:46.433578 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] The token cache is being invalidate because project 83c238b7deb4427b92d7d225a04b2739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:41:46.433865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.434494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8a7dbcd2-48e4-4ad5-8b5e-062144cac015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:46.436580 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1418/2815] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:41:46 2022] DELETE /identity/v3/projects/83c238b7deb4427b92d7d225a04b2739 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:46.441686 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984d52d-48c4-48cf-8d40-21372da1cc0c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.442175 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984d52d-48c4-48cf-8d40-21372da1cc0c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.442313 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984d52d-48c4-48cf-8d40-21372da1cc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.461237 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6984d52d-48c4-48cf-8d40-21372da1cc0c None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:46.462424 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6984d52d-48c4-48cf-8d40-21372da1cc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:46.506289 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:46.522988 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1398/2816] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:46.531095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21e15-04d5-42a9-8694-dd6d21bb92cb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.531415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21e15-04d5-42a9-8694-dd6d21bb92cb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.531724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21e15-04d5-42a9-8694-dd6d21bb92cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.551185 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0ae21e15-04d5-42a9-8694-dd6d21bb92cb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:46.552413 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0ae21e15-04d5-42a9-8694-dd6d21bb92cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:46.561102 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1419/2817] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:46.564964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984d52d-48c4-48cf-8d40-21372da1cc0c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:46.574561 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984d52d-48c4-48cf-8d40-21372da1cc0c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:46.579251 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:46.579938 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.580223 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.580497 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.581221 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:41:46.581557 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:46.583140 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:46.594532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1399/2818] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:41:46.598666 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae21e15-04d5-42a9-8694-dd6d21bb92cb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:46.609230 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae21e15-04d5-42a9-8694-dd6d21bb92cb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:46.613935 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525dc11-31ac-41ca-b605-a4a8478eff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:46.614577 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525dc11-31ac-41ca-b605-a4a8478eff8f admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.614865 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525dc11-31ac-41ca-b605-a4a8478eff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.615132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525dc11-31ac-41ca-b605-a4a8478eff8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.615861 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525dc11-31ac-41ca-b605-a4a8478eff8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:46.617747 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525dc11-31ac-41ca-b605-a4a8478eff8f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:46.643327 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1420/2819] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:41:46 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:46.647525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:46.658064 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e5b7a-205c-4d11-8a04-08f854d86f7e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:46.716335 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ca485-a017-4fae-9d53-a7d293dc2e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:46.717106 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca485-a017-4fae-9d53-a7d293dc2e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.717465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca485-a017-4fae-9d53-a7d293dc2e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.717813 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca485-a017-4fae-9d53-a7d293dc2e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.718524 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ca485-a017-4fae-9d53-a7d293dc2e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:46.720895 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ca485-a017-4fae-9d53-a7d293dc2e63 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:46.741654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1400/2820] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:41:46 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:46.745758 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525dc11-31ac-41ca-b605-a4a8478eff8f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:46.757289 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525dc11-31ac-41ca-b605-a4a8478eff8f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:46.761934 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262766f-4017-495e-b8c1-976c435b5132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:46.762600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262766f-4017-495e-b8c1-976c435b5132 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.762884 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262766f-4017-495e-b8c1-976c435b5132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.763310 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262766f-4017-495e-b8c1-976c435b5132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.764252 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262766f-4017-495e-b8c1-976c435b5132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:46.765888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262766f-4017-495e-b8c1-976c435b5132 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:46.776246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1421/2821] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:46.780167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ca485-a017-4fae-9d53-a7d293dc2e63 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:46.790492 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ca485-a017-4fae-9d53-a7d293dc2e63 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:46.795862 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f3e2e-77e3-495d-ad5a-bf0fb3d20948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:46.796521 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3e2e-77e3-495d-ad5a-bf0fb3d20948 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.796809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3e2e-77e3-495d-ad5a-bf0fb3d20948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.797092 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3e2e-77e3-495d-ad5a-bf0fb3d20948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.797817 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f3e2e-77e3-495d-ad5a-bf0fb3d20948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:46.799439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f3e2e-77e3-495d-ad5a-bf0fb3d20948 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:46.809621 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1401/2822] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:46.814704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262766f-4017-495e-b8c1-976c435b5132 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:46.824557 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262766f-4017-495e-b8c1-976c435b5132 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:46.829783 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd66a5-873b-4234-9e75-aff61741dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:46.830654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd66a5-873b-4234-9e75-aff61741dbee admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.830932 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd66a5-873b-4234-9e75-aff61741dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.831204 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd66a5-873b-4234-9e75-aff61741dbee admin admin] PATH_INFO: `/v3/projects/5c56ed52275d4bd2b35317678a768e87/users/aeb635f1b1884a2189c57c412817d277/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.845673 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd66a5-873b-4234-9e75-aff61741dbee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c56ed52275d4bd2b35317678a768e87, user_id=aeb635f1b1884a2189c57c412817d277, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:46.848363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd66a5-873b-4234-9e75-aff61741dbee admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:46.861807 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1422/2823] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:46 2022] PUT /identity/v3/projects/5c56ed52275d4bd2b35317678a768e87/users/aeb635f1b1884a2189c57c412817d277/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:46.873620 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2520a6-7984-4044-b6ef-7716ff238325 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:46.873919 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2520a6-7984-4044-b6ef-7716ff238325 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:46.874193 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2520a6-7984-4044-b6ef-7716ff238325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:46.900312 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cd2520a6-7984-4044-b6ef-7716ff238325 None None] MFA Rules not processed for user `aeb635f1b1884a2189c57c412817d277`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:46.901500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cd2520a6-7984-4044-b6ef-7716ff238325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:46.965182 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1402/2824] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:46.970295 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd66a5-873b-4234-9e75-aff61741dbee admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.032429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd66a5-873b-4234-9e75-aff61741dbee admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.037171 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca1b9e-42ab-4ab3-8011-a7bb8f355303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.037829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1b9e-42ab-4ab3-8011-a7bb8f355303 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.038112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1b9e-42ab-4ab3-8011-a7bb8f355303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.038419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1b9e-42ab-4ab3-8011-a7bb8f355303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.039108 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca1b9e-42ab-4ab3-8011-a7bb8f355303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.041067 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca1b9e-42ab-4ab3-8011-a7bb8f355303 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.066074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1423/2825] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:41:46 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:47.070070 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2520a6-7984-4044-b6ef-7716ff238325 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.081162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2520a6-7984-4044-b6ef-7716ff238325 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.122622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:47.138231 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09b460-899c-4043-a510-6715bddf0087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.139084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b460-899c-4043-a510-6715bddf0087 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.139389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b460-899c-4043-a510-6715bddf0087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.139668 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b460-899c-4043-a510-6715bddf0087 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.140357 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09b460-899c-4043-a510-6715bddf0087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.142254 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09b460-899c-4043-a510-6715bddf0087 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.161708 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1403/2826] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:41:47 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:41:47.165745 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca1b9e-42ab-4ab3-8011-a7bb8f355303 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.176470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca1b9e-42ab-4ab3-8011-a7bb8f355303 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.181000 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645269-93f9-41b4-b61a-96ba2de688dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.181642 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645269-93f9-41b4-b61a-96ba2de688dc admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.181923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645269-93f9-41b4-b61a-96ba2de688dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.182194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645269-93f9-41b4-b61a-96ba2de688dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.182911 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645269-93f9-41b4-b61a-96ba2de688dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.184867 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645269-93f9-41b4-b61a-96ba2de688dc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.195287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1424/2827] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:47.199209 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09b460-899c-4043-a510-6715bddf0087 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.209028 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09b460-899c-4043-a510-6715bddf0087 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.213618 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb1a1a-830d-4c51-b4bc-0fceb8310c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.214493 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1a1a-830d-4c51-b4bc-0fceb8310c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.214774 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1a1a-830d-4c51-b4bc-0fceb8310c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.215051 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1a1a-830d-4c51-b4bc-0fceb8310c4a admin admin] PATH_INFO: `/v3/projects/e255daf4735c46209e31103aebedfed1/users/2d141a7681ea45149c9f6dbefc186841/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.229706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb1a1a-830d-4c51-b4bc-0fceb8310c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e255daf4735c46209e31103aebedfed1, user_id=2d141a7681ea45149c9f6dbefc186841, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.233014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb1a1a-830d-4c51-b4bc-0fceb8310c4a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.245297 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1404/2828] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:47 2022] PUT /identity/v3/projects/e255daf4735c46209e31103aebedfed1/users/2d141a7681ea45149c9f6dbefc186841/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:47.249134 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645269-93f9-41b4-b61a-96ba2de688dc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.259454 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645269-93f9-41b4-b61a-96ba2de688dc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.314586 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c54a96-912c-45ab-9958-a30a18bf6c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.315243 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c54a96-912c-45ab-9958-a30a18bf6c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.315553 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c54a96-912c-45ab-9958-a30a18bf6c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.315828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c54a96-912c-45ab-9958-a30a18bf6c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.316547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c54a96-912c-45ab-9958-a30a18bf6c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.318142 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c54a96-912c-45ab-9958-a30a18bf6c5f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.323071 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:47.329819 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1425/2829] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:47.333572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb1a1a-830d-4c51-b4bc-0fceb8310c4a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.344788 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb1a1a-830d-4c51-b4bc-0fceb8310c4a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.349401 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.350274 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.350556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.350833 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] PATH_INFO: `/v3/projects/e255daf4735c46209e31103aebedfed1/users/2d141a7681ea45149c9f6dbefc186841/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.352811 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e255daf4735c46209e31103aebedfed1, user_id=2d141a7681ea45149c9f6dbefc186841, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.355532 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.365683 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1405/2830] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:47 2022] PUT /identity/v3/projects/e255daf4735c46209e31103aebedfed1/users/2d141a7681ea45149c9f6dbefc186841/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:47.375981 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6449bf2-b548-4a89-95bc-24be686bff79 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.376417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6449bf2-b548-4a89-95bc-24be686bff79 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.376779 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6449bf2-b548-4a89-95bc-24be686bff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.405631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f6449bf2-b548-4a89-95bc-24be686bff79 None None] MFA Rules not processed for user `2d141a7681ea45149c9f6dbefc186841`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:47.406870 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f6449bf2-b548-4a89-95bc-24be686bff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:47.473202 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1426/2831] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:47.485032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.486840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:47.543284 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b9b6e-a245-415d-80d2-6c81365ea08c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.547084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1226dce-0526-4667-b10e-b0b933202666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.547749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1226dce-0526-4667-b10e-b0b933202666 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.548031 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1226dce-0526-4667-b10e-b0b933202666 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.548307 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1226dce-0526-4667-b10e-b0b933202666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.557060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1226dce-0526-4667-b10e-b0b933202666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.558475 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1226dce-0526-4667-b10e-b0b933202666 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.567863 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1406/2832] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:47.596560 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6449bf2-b548-4a89-95bc-24be686bff79 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.658830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6449bf2-b548-4a89-95bc-24be686bff79 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.663325 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.664214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.664496 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.664767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] PATH_INFO: `/v3/users/aeb635f1b1884a2189c57c412817d277` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.665960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb635f1b1884a2189c57c412817d277)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.667915 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.713665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb635f1b1884a2189c57c412817d277'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:47.714381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb635f1b1884a2189c57c412817d277'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:47.722027 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb635f1b1884a2189c57c412817d277'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:47.733681 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb635f1b1884a2189c57c412817d277'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:47.734370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb635f1b1884a2189c57c412817d277'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:47.747093 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1427/2833] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:41:47 2022] DELETE /identity/v3/users/aeb635f1b1884a2189c57c412817d277 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:47.752547 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b636-ae76-4af7-9fbb-7b1cdfa66a74 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.752837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b636-ae76-4af7-9fbb-7b1cdfa66a74 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.753115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b636-ae76-4af7-9fbb-7b1cdfa66a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.770859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f881b636-ae76-4af7-9fbb-7b1cdfa66a74 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:47.772089 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f881b636-ae76-4af7-9fbb-7b1cdfa66a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:47.832060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1407/2834] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:47.841664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:47.843049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:47.849509 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:47.901668 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73755276-1623-4f12-ab0d-d09b08f4d4a1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:47.905533 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2026-0989-4e01-a490-70903ca9ee0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:47.906208 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2026-0989-4e01-a490-70903ca9ee0e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:47.906491 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2026-0989-4e01-a490-70903ca9ee0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:47.906769 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2026-0989-4e01-a490-70903ca9ee0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:47.916006 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d2026-0989-4e01-a490-70903ca9ee0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:47.917364 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d2026-0989-4e01-a490-70903ca9ee0e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:47.926864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1428/2835] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:48.193116 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881b636-ae76-4af7-9fbb-7b1cdfa66a74 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:48.194555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f881b636-ae76-4af7-9fbb-7b1cdfa66a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:48.210230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881b636-ae76-4af7-9fbb-7b1cdfa66a74 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:48.214849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:48.215763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:48.216047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:48.216591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] PATH_INFO: `/v3/projects/5c56ed52275d4bd2b35317678a768e87` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:48.217818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c56ed52275d4bd2b35317678a768e87)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:48.219916 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:48.257850 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:48.269258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c56ed52275d4bd2b35317678a768e87', 'request_id': 'req-bc71328d-467c-4fa8-9475-b06e2d7285da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.276059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c56ed52275d4bd2b35317678a768e87', 'request_id': 'req-bc71328d-467c-4fa8-9475-b06e2d7285da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.282272 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c56ed52275d4bd2b35317678a768e87', 'request_id': 'req-bc71328d-467c-4fa8-9475-b06e2d7285da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.292555 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] The token cache is being invalidate because project 5c56ed52275d4bd2b35317678a768e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:41:48.292840 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.293489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.295679 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1408/2836] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:41:48 2022] DELETE /identity/v3/projects/5c56ed52275d4bd2b35317678a768e87 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:41:48.299969 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2026-0989-4e01-a490-70903ca9ee0e service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:48.309482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2026-0989-4e01-a490-70903ca9ee0e service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:48.365621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:48.366486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:48.366767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:48.367043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] PATH_INFO: `/v3/users/2d141a7681ea45149c9f6dbefc186841` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:48.368263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d141a7681ea45149c9f6dbefc186841)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:48.370195 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:48.416020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d141a7681ea45149c9f6dbefc186841'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.416648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d141a7681ea45149c9f6dbefc186841'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.423652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d141a7681ea45149c9f6dbefc186841'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.435927 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d141a7681ea45149c9f6dbefc186841'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.436510 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-03b16485-d219-4525-81f0-09c7e8b4f70b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d141a7681ea45149c9f6dbefc186841'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.449084 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1429/2837] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:41:48 2022] DELETE /identity/v3/users/2d141a7681ea45149c9f6dbefc186841 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:48.716978 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:48.719375 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:48.735227 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71328d-467c-4fa8-9475-b06e2d7285da admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:48.792066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:48.793117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:48.793400 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:48.793671 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] PATH_INFO: `/v3/projects/e255daf4735c46209e31103aebedfed1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:48.794987 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=e255daf4735c46209e31103aebedfed1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:48.797150 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:48.845059 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e255daf4735c46209e31103aebedfed1', 'request_id': 'req-0c793999-6c8a-4042-9665-1a44941813ea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.851761 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e255daf4735c46209e31103aebedfed1', 'request_id': 'req-0c793999-6c8a-4042-9665-1a44941813ea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.858230 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e255daf4735c46209e31103aebedfed1', 'request_id': 'req-0c793999-6c8a-4042-9665-1a44941813ea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.868482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] The token cache is being invalidate because project e255daf4735c46209e31103aebedfed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:41:48.868769 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.869403 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c793999-6c8a-4042-9665-1a44941813ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:48.870967 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1409/2838] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:41:48 2022] DELETE /identity/v3/projects/e255daf4735c46209e31103aebedfed1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:48.876219 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ccdf6-5f2d-4b0c-9765-79e99eb9e898 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:48.876767 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ccdf6-5f2d-4b0c-9765-79e99eb9e898 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:48.877048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ccdf6-5f2d-4b0c-9765-79e99eb9e898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:48.894909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-020ccdf6-5f2d-4b0c-9765-79e99eb9e898 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:48.896153 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-020ccdf6-5f2d-4b0c-9765-79e99eb9e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:48.917112 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:48.956234 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1430/2839] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:48.963872 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddd24b-d08d-4af1-921c-57c1dfd0543e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:48.964098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddd24b-d08d-4af1-921c-57c1dfd0543e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:48.964371 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddd24b-d08d-4af1-921c-57c1dfd0543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:48.983084 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-eeddd24b-d08d-4af1-921c-57c1dfd0543e None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:48.984300 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-eeddd24b-d08d-4af1-921c-57c1dfd0543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:48.992040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1410/2840] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:48.996048 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ccdf6-5f2d-4b0c-9765-79e99eb9e898 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.006173 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ccdf6-5f2d-4b0c-9765-79e99eb9e898 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.010668 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.011309 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.011618 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.011891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.012608 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:41:49.013436 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.015330 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.026538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1431/2841] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:41:49.031697 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddd24b-d08d-4af1-921c-57c1dfd0543e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.040827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddd24b-d08d-4af1-921c-57c1dfd0543e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.045636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfe98f-882d-4b02-bea3-3a39102d29e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.046290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe98f-882d-4b02-bea3-3a39102d29e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.046572 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe98f-882d-4b02-bea3-3a39102d29e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.046850 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe98f-882d-4b02-bea3-3a39102d29e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.047576 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfe98f-882d-4b02-bea3-3a39102d29e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.049470 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfe98f-882d-4b02-bea3-3a39102d29e3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.074361 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1411/2842] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:41:49 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:41:49.078829 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.088970 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44d670-d526-482f-a37b-4e05c8cc738e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.151451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2235701-898c-4d24-8c3c-7c011819eccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.152113 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2235701-898c-4d24-8c3c-7c011819eccb admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.152394 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2235701-898c-4d24-8c3c-7c011819eccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.152665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2235701-898c-4d24-8c3c-7c011819eccb admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.153365 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2235701-898c-4d24-8c3c-7c011819eccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.156273 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2235701-898c-4d24-8c3c-7c011819eccb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.176650 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1432/2843] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:41:49 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:49.180759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfe98f-882d-4b02-bea3-3a39102d29e3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.191940 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfe98f-882d-4b02-bea3-3a39102d29e3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.196482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3048c25-c0a9-4946-b880-4e7279958b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.197133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3048c25-c0a9-4946-b880-4e7279958b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.197415 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3048c25-c0a9-4946-b880-4e7279958b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.197691 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3048c25-c0a9-4946-b880-4e7279958b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.198410 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3048c25-c0a9-4946-b880-4e7279958b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.200028 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3048c25-c0a9-4946-b880-4e7279958b9d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.210896 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1412/2844] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:49.214825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2235701-898c-4d24-8c3c-7c011819eccb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.224563 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2235701-898c-4d24-8c3c-7c011819eccb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.229083 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3357f-0e84-4bd9-b571-dbeb56803784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.229732 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3357f-0e84-4bd9-b571-dbeb56803784 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.230014 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3357f-0e84-4bd9-b571-dbeb56803784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.230292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3357f-0e84-4bd9-b571-dbeb56803784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.231013 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3357f-0e84-4bd9-b571-dbeb56803784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.232752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3357f-0e84-4bd9-b571-dbeb56803784 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.242411 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1433/2845] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:49.246199 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3048c25-c0a9-4946-b880-4e7279958b9d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.256888 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3048c25-c0a9-4946-b880-4e7279958b9d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.261378 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1af3-f683-46da-9075-fed80e6136f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.262238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1af3-f683-46da-9075-fed80e6136f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.262519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1af3-f683-46da-9075-fed80e6136f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.262792 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1af3-f683-46da-9075-fed80e6136f4 admin admin] PATH_INFO: `/v3/projects/8460a3e41eae4bba9c61941f976bb4a6/users/05e18bdbcd134382af72d66ed3d07297/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.274113 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:49.278964 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f1af3-f683-46da-9075-fed80e6136f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8460a3e41eae4bba9c61941f976bb4a6, user_id=05e18bdbcd134382af72d66ed3d07297, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.282222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f1af3-f683-46da-9075-fed80e6136f4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.295520 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1413/2846] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:49 2022] PUT /identity/v3/projects/8460a3e41eae4bba9c61941f976bb4a6/users/05e18bdbcd134382af72d66ed3d07297/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:49.306781 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d6548-5221-4d72-bfdc-a95dc844268e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.307078 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d6548-5221-4d72-bfdc-a95dc844268e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.307392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d6548-5221-4d72-bfdc-a95dc844268e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.334441 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-da2d6548-5221-4d72-bfdc-a95dc844268e None None] MFA Rules not processed for user `05e18bdbcd134382af72d66ed3d07297`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:49.335659 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-da2d6548-5221-4d72-bfdc-a95dc844268e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:49.399282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1434/2847] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:49.403834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1af3-f683-46da-9075-fed80e6136f4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.467120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1af3-f683-46da-9075-fed80e6136f4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.471634 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d7406-8c43-4906-a4a9-09aa18fb65ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.472290 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7406-8c43-4906-a4a9-09aa18fb65ab admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.472569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7406-8c43-4906-a4a9-09aa18fb65ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.472841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7406-8c43-4906-a4a9-09aa18fb65ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.473525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d7406-8c43-4906-a4a9-09aa18fb65ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.475431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d7406-8c43-4906-a4a9-09aa18fb65ab admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.500488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1414/2848] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:41:49 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:49.504796 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d6548-5221-4d72-bfdc-a95dc844268e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.517138 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d6548-5221-4d72-bfdc-a95dc844268e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.535663 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:49.573891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817dd04-2714-4e9d-b4af-82cce526a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.574554 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817dd04-2714-4e9d-b4af-82cce526a192 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.574831 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817dd04-2714-4e9d-b4af-82cce526a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.575103 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817dd04-2714-4e9d-b4af-82cce526a192 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.576400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817dd04-2714-4e9d-b4af-82cce526a192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.578270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817dd04-2714-4e9d-b4af-82cce526a192 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.600094 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1435/2849] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:41:49 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:41:49.601844 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d7406-8c43-4906-a4a9-09aa18fb65ab admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.612315 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d7406-8c43-4906-a4a9-09aa18fb65ab admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.616898 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724b313-26b0-4559-9412-5f457edfdc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.617550 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724b313-26b0-4559-9412-5f457edfdc4d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.617834 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724b313-26b0-4559-9412-5f457edfdc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.618109 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724b313-26b0-4559-9412-5f457edfdc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.618829 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724b313-26b0-4559-9412-5f457edfdc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.620439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724b313-26b0-4559-9412-5f457edfdc4d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.631220 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1415/2850] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:49.635250 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817dd04-2714-4e9d-b4af-82cce526a192 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.645921 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817dd04-2714-4e9d-b4af-82cce526a192 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.650476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5be21b-03f0-4f69-8893-ed9eeaa0f719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.651367 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5be21b-03f0-4f69-8893-ed9eeaa0f719 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.651653 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5be21b-03f0-4f69-8893-ed9eeaa0f719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.651920 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5be21b-03f0-4f69-8893-ed9eeaa0f719 admin admin] PATH_INFO: `/v3/projects/3e4b2959e43747dfb92f2b4d4c83c04d/users/04a8e503b15d47e7a7933bc0e4beca6b/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.667043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5be21b-03f0-4f69-8893-ed9eeaa0f719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4b2959e43747dfb92f2b4d4c83c04d, user_id=04a8e503b15d47e7a7933bc0e4beca6b, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.670137 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5be21b-03f0-4f69-8893-ed9eeaa0f719 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.682498 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1436/2851] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:49 2022] PUT /identity/v3/projects/3e4b2959e43747dfb92f2b4d4c83c04d/users/04a8e503b15d47e7a7933bc0e4beca6b/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:41:49.686598 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724b313-26b0-4559-9412-5f457edfdc4d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.696367 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724b313-26b0-4559-9412-5f457edfdc4d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.753036 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9e43c-077f-41b0-89bd-91da6d2a18b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.753701 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9e43c-077f-41b0-89bd-91da6d2a18b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.753982 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9e43c-077f-41b0-89bd-91da6d2a18b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.754258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9e43c-077f-41b0-89bd-91da6d2a18b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.754973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9e43c-077f-41b0-89bd-91da6d2a18b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.756589 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9e43c-077f-41b0-89bd-91da6d2a18b6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.766828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1416/2852] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:49.771069 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5be21b-03f0-4f69-8893-ed9eeaa0f719 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.782064 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5be21b-03f0-4f69-8893-ed9eeaa0f719 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.786700 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.787609 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.787892 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.788168 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] PATH_INFO: `/v3/projects/3e4b2959e43747dfb92f2b4d4c83c04d/users/04a8e503b15d47e7a7933bc0e4beca6b/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.790131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4b2959e43747dfb92f2b4d4c83c04d, user_id=04a8e503b15d47e7a7933bc0e4beca6b, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.792813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:49.802958 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1437/2853] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:49 2022] PUT /identity/v3/projects/3e4b2959e43747dfb92f2b4d4c83c04d/users/04a8e503b15d47e7a7933bc0e4beca6b/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:49.812965 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60ebe1-b2fa-43f3-a079-2fc0919301ea None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.813256 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60ebe1-b2fa-43f3-a079-2fc0919301ea None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.813533 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60ebe1-b2fa-43f3-a079-2fc0919301ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.839186 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0a60ebe1-b2fa-43f3-a079-2fc0919301ea None None] MFA Rules not processed for user `04a8e503b15d47e7a7933bc0e4beca6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:49.840402 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0a60ebe1-b2fa-43f3-a079-2fc0919301ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:49.886805 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:49.909864 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1417/2854] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:49.921741 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:49.923503 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:49.980502 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56f26f-aba8-4195-b965-80913a671940 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:49.984320 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cc556-234a-494b-a77d-dd598d7f5852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:49.984961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc556-234a-494b-a77d-dd598d7f5852 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:49.985244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc556-234a-494b-a77d-dd598d7f5852 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:49.985520 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc556-234a-494b-a77d-dd598d7f5852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:49.994421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cc556-234a-494b-a77d-dd598d7f5852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:49.995891 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cc556-234a-494b-a77d-dd598d7f5852 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:50.006403 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1438/2855] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:49 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:50.090667 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60ebe1-b2fa-43f3-a079-2fc0919301ea None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:50.100488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60ebe1-b2fa-43f3-a079-2fc0919301ea None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:50.104280 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca0e68-1c0f-4ba7-8e6b-6eca65fbba51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:50.104920 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca0e68-1c0f-4ba7-8e6b-6eca65fbba51 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:50.105201 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca0e68-1c0f-4ba7-8e6b-6eca65fbba51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:50.105481 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca0e68-1c0f-4ba7-8e6b-6eca65fbba51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:50.163287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca0e68-1c0f-4ba7-8e6b-6eca65fbba51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:50.164751 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca0e68-1c0f-4ba7-8e6b-6eca65fbba51 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:50.173594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1418/2856] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:54.825320 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cc556-234a-494b-a77d-dd598d7f5852 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:54.890361 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cc556-234a-494b-a77d-dd598d7f5852 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:54.895163 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:54.896066 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:54.896341 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:54.896617 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] PATH_INFO: `/v3/users/05e18bdbcd134382af72d66ed3d07297` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:54.898021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e18bdbcd134382af72d66ed3d07297)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:54.900001 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:54.922247 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:54.948768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e18bdbcd134382af72d66ed3d07297'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:54.949415 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e18bdbcd134382af72d66ed3d07297'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:54.956223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e18bdbcd134382af72d66ed3d07297'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:54.967965 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e18bdbcd134382af72d66ed3d07297'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:54.968585 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e18bdbcd134382af72d66ed3d07297'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:54.981740 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1439/2857] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:41:54 2022] DELETE /identity/v3/users/05e18bdbcd134382af72d66ed3d07297 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:54.986759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c03d60-4e35-4407-b14d-1e6fae880be7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:54.987258 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c03d60-4e35-4407-b14d-1e6fae880be7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:54.987394 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c03d60-4e35-4407-b14d-1e6fae880be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:55.005795 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-68c03d60-4e35-4407-b14d-1e6fae880be7 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:55.007038 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-68c03d60-4e35-4407-b14d-1e6fae880be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:55.068082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1419/2858] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:41:55.078119 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:55.079538 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:55.137285 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173d216-fc67-4d44-9d25-8aaa0fe5f93f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:55.141229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c3163-9cd3-4279-859e-4686e2882f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:55.141864 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c3163-9cd3-4279-859e-4686e2882f0d service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:55.142144 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c3163-9cd3-4279-859e-4686e2882f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:55.142417 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c3163-9cd3-4279-859e-4686e2882f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:55.151176 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c3163-9cd3-4279-859e-4686e2882f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:55.152631 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c3163-9cd3-4279-859e-4686e2882f0d service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:55.162074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1440/2859] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:41:55.453803 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c03d60-4e35-4407-b14d-1e6fae880be7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:55.455505 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-68c03d60-4e35-4407-b14d-1e6fae880be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:55.468759 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c03d60-4e35-4407-b14d-1e6fae880be7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:55.473506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:55.474372 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:55.474648 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:55.474918 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] PATH_INFO: `/v3/projects/8460a3e41eae4bba9c61941f976bb4a6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:55.476117 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8460a3e41eae4bba9c61941f976bb4a6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:55.478061 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:55.526187 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8460a3e41eae4bba9c61941f976bb4a6', 'request_id': 'req-5685cb2b-b032-49c0-ab13-66d6e23b1543'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.536721 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8460a3e41eae4bba9c61941f976bb4a6', 'request_id': 'req-5685cb2b-b032-49c0-ab13-66d6e23b1543'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.543142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8460a3e41eae4bba9c61941f976bb4a6', 'request_id': 'req-5685cb2b-b032-49c0-ab13-66d6e23b1543'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.555839 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] The token cache is being invalidate because project 8460a3e41eae4bba9c61941f976bb4a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:41:55.556125 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.556776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.566451 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1420/2860] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:41:55 2022] DELETE /identity/v3/projects/8460a3e41eae4bba9c61941f976bb4a6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:55.570421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c3163-9cd3-4279-859e-4686e2882f0d service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:55.581029 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c3163-9cd3-4279-859e-4686e2882f0d service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:55.639053 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:55.639953 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:55.640235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:55.640514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] PATH_INFO: `/v3/users/04a8e503b15d47e7a7933bc0e4beca6b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:55.641711 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a8e503b15d47e7a7933bc0e4beca6b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:55.643678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:55.685646 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:55.691931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a8e503b15d47e7a7933bc0e4beca6b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.692706 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a8e503b15d47e7a7933bc0e4beca6b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.700319 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a8e503b15d47e7a7933bc0e4beca6b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.712749 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a8e503b15d47e7a7933bc0e4beca6b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.713421 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-75a8fae5-3fe8-434d-9174-09e59c7d65f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a8e503b15d47e7a7933bc0e4beca6b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:55.730479 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1441/2861] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:41:55 2022] DELETE /identity/v3/users/04a8e503b15d47e7a7933bc0e4beca6b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:55.996461 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:55.998836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:56.012146 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685cb2b-b032-49c0-ab13-66d6e23b1543 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.016594 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:56.073763 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.074623 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.074902 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.075174 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] PATH_INFO: `/v3/projects/3e4b2959e43747dfb92f2b4d4c83c04d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.077011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4b2959e43747dfb92f2b4d4c83c04d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.078900 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.123854 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4b2959e43747dfb92f2b4d4c83c04d', 'request_id': 'req-d64236f0-a6ae-4546-a330-e62c3bf18401'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:56.131090 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4b2959e43747dfb92f2b4d4c83c04d', 'request_id': 'req-d64236f0-a6ae-4546-a330-e62c3bf18401'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:56.137283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4b2959e43747dfb92f2b4d4c83c04d', 'request_id': 'req-d64236f0-a6ae-4546-a330-e62c3bf18401'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:56.148005 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] The token cache is being invalidate because project 3e4b2959e43747dfb92f2b4d4c83c04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:41:56.148276 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:56.148909 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d64236f0-a6ae-4546-a330-e62c3bf18401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:41:56.150439 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1421/2862] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:41:55 2022] DELETE /identity/v3/projects/3e4b2959e43747dfb92f2b4d4c83c04d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:56.157637 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623c1a6-c636-4f6d-a33e-e01943201777 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.158144 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623c1a6-c636-4f6d-a33e-e01943201777 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.158249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623c1a6-c636-4f6d-a33e-e01943201777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.179680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2623c1a6-c636-4f6d-a33e-e01943201777 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:56.180890 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2623c1a6-c636-4f6d-a33e-e01943201777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:56.241871 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1442/2863] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:41:56.250162 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073ab25-6c42-45b0-9a09-1149716a6ac9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.250449 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073ab25-6c42-45b0-9a09-1149716a6ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.250744 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073ab25-6c42-45b0-9a09-1149716a6ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.269631 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1073ab25-6c42-45b0-9a09-1149716a6ac9 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:56.270828 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1073ab25-6c42-45b0-9a09-1149716a6ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:56.278749 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1422/2864] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:56.283183 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623c1a6-c636-4f6d-a33e-e01943201777 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.292775 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623c1a6-c636-4f6d-a33e-e01943201777 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.297923 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.298572 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.298853 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.299127 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.299897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:41:56.300206 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.301820 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.314032 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1443/2865] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:41:56.318482 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073ab25-6c42-45b0-9a09-1149716a6ac9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.327892 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073ab25-6c42-45b0-9a09-1149716a6ac9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.332569 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fee235-1f59-40a9-a628-d8f475f7ea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.333222 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fee235-1f59-40a9-a628-d8f475f7ea55 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.333506 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fee235-1f59-40a9-a628-d8f475f7ea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.333784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fee235-1f59-40a9-a628-d8f475f7ea55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.334484 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fee235-1f59-40a9-a628-d8f475f7ea55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.336440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fee235-1f59-40a9-a628-d8f475f7ea55 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.362173 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1423/2866] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:41:56 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 17:41:56.366229 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.379040 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36f30-726c-40c0-bcb9-a6f1435d3354 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.434486 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde1dd1-6cbc-4479-bcf3-d8c58419f2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.435830 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde1dd1-6cbc-4479-bcf3-d8c58419f2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.436292 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde1dd1-6cbc-4479-bcf3-d8c58419f2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.436574 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde1dd1-6cbc-4479-bcf3-d8c58419f2f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.437275 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde1dd1-6cbc-4479-bcf3-d8c58419f2f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.439190 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde1dd1-6cbc-4479-bcf3-d8c58419f2f9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.459656 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1444/2867] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:41:56 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:56.463635 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fee235-1f59-40a9-a628-d8f475f7ea55 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.474554 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fee235-1f59-40a9-a628-d8f475f7ea55 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.479479 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6e462-e8af-4183-935e-041ebceba2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.480149 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6e462-e8af-4183-935e-041ebceba2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.480431 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6e462-e8af-4183-935e-041ebceba2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.481194 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6e462-e8af-4183-935e-041ebceba2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.481513 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6e462-e8af-4183-935e-041ebceba2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.483959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6e462-e8af-4183-935e-041ebceba2a0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.494445 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1424/2868] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:56.499541 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde1dd1-6cbc-4479-bcf3-d8c58419f2f9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.509691 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde1dd1-6cbc-4479-bcf3-d8c58419f2f9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.514257 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87beb34-819e-4b4b-a16e-af6e4a44ebf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.514908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87beb34-819e-4b4b-a16e-af6e4a44ebf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.515188 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87beb34-819e-4b4b-a16e-af6e4a44ebf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.515500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87beb34-819e-4b4b-a16e-af6e4a44ebf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.516225 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87beb34-819e-4b4b-a16e-af6e4a44ebf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.517809 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87beb34-819e-4b4b-a16e-af6e4a44ebf5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.527546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1445/2869] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 17:41:56.531890 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6e462-e8af-4183-935e-041ebceba2a0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.541731 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6e462-e8af-4183-935e-041ebceba2a0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.546908 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b95008-b139-4edd-af40-e259b6d199f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.547820 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95008-b139-4edd-af40-e259b6d199f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.548104 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95008-b139-4edd-af40-e259b6d199f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.548379 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95008-b139-4edd-af40-e259b6d199f0 admin admin] PATH_INFO: `/v3/projects/636d745a71364d3a92df90e19d438811/users/f84fca1d74de49f48a7f629ef9fb54a9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.563142 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b95008-b139-4edd-af40-e259b6d199f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636d745a71364d3a92df90e19d438811, user_id=f84fca1d74de49f48a7f629ef9fb54a9, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.565806 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b95008-b139-4edd-af40-e259b6d199f0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.579011 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1425/2870] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:56 2022] PUT /identity/v3/projects/636d745a71364d3a92df90e19d438811/users/f84fca1d74de49f48a7f629ef9fb54a9/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:56.641115 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3c8c-5575-4a8d-850b-c320c17cdccb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.641402 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3c8c-5575-4a8d-850b-c320c17cdccb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.641682 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3c8c-5575-4a8d-850b-c320c17cdccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.667710 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-602c3c8c-5575-4a8d-850b-c320c17cdccb None None] MFA Rules not processed for user `f84fca1d74de49f48a7f629ef9fb54a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:56.668917 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-602c3c8c-5575-4a8d-850b-c320c17cdccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:56.680909 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:56.738248 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1446/2871] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:56.742706 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b95008-b139-4edd-af40-e259b6d199f0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.775921 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:56.805503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b95008-b139-4edd-af40-e259b6d199f0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.810334 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d71ce-7b47-43c9-9852-5475482947d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.811008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d71ce-7b47-43c9-9852-5475482947d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.811293 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d71ce-7b47-43c9-9852-5475482947d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.811604 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d71ce-7b47-43c9-9852-5475482947d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.812303 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d71ce-7b47-43c9-9852-5475482947d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.814197 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d71ce-7b47-43c9-9852-5475482947d7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.838396 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1426/2872] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:41:56 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:56.843049 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c3c8c-5575-4a8d-850b-c320c17cdccb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.853748 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c3c8c-5575-4a8d-850b-c320c17cdccb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.910244 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df742b-77b0-4bf4-90c0-7dcd88402f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.910908 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df742b-77b0-4bf4-90c0-7dcd88402f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.911194 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df742b-77b0-4bf4-90c0-7dcd88402f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.911514 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df742b-77b0-4bf4-90c0-7dcd88402f3b admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.912215 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df742b-77b0-4bf4-90c0-7dcd88402f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.914137 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df742b-77b0-4bf4-90c0-7dcd88402f3b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.933966 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1447/2873] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:41:56 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:41:56.937825 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d71ce-7b47-43c9-9852-5475482947d7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.947877 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d71ce-7b47-43c9-9852-5475482947d7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.952767 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087292cd-c055-4976-ada8-b6581944988f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.953423 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087292cd-c055-4976-ada8-b6581944988f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.953703 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087292cd-c055-4976-ada8-b6581944988f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.953979 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087292cd-c055-4976-ada8-b6581944988f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:56.954700 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087292cd-c055-4976-ada8-b6581944988f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:56.956310 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087292cd-c055-4976-ada8-b6581944988f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:56.967433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1427/2874] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:56.971501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df742b-77b0-4bf4-90c0-7dcd88402f3b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:56.981120 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df742b-77b0-4bf4-90c0-7dcd88402f3b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:56.986648 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477d1f4-066e-4cf0-9772-057bdebceca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:56.987547 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477d1f4-066e-4cf0-9772-057bdebceca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:56.987824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477d1f4-066e-4cf0-9772-057bdebceca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:56.988095 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477d1f4-066e-4cf0-9772-057bdebceca6 admin admin] PATH_INFO: `/v3/projects/ce46a02781c74eb5a27d945b8f1b8384/users/ea1954ef80404716929008b389c9e990/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:57.002897 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477d1f4-066e-4cf0-9772-057bdebceca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce46a02781c74eb5a27d945b8f1b8384, user_id=ea1954ef80404716929008b389c9e990, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:57.005559 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477d1f4-066e-4cf0-9772-057bdebceca6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:57.017910 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1448/2875] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:56 2022] PUT /identity/v3/projects/ce46a02781c74eb5a27d945b8f1b8384/users/ea1954ef80404716929008b389c9e990/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 17:41:57.021791 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087292cd-c055-4976-ada8-b6581944988f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:57.032859 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087292cd-c055-4976-ada8-b6581944988f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:57.087809 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d12c60-31ff-4c7d-a339-60261e07fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:57.088496 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d12c60-31ff-4c7d-a339-60261e07fac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:57.088780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d12c60-31ff-4c7d-a339-60261e07fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:57.089058 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d12c60-31ff-4c7d-a339-60261e07fac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:57.089780 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d12c60-31ff-4c7d-a339-60261e07fac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:57.091989 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d12c60-31ff-4c7d-a339-60261e07fac7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:57.101640 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1428/2876] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:41:57.105739 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477d1f4-066e-4cf0-9772-057bdebceca6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:57.116018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477d1f4-066e-4cf0-9772-057bdebceca6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:57.120528 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:57.121401 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:57.121680 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:57.121952 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] PATH_INFO: `/v3/projects/ce46a02781c74eb5a27d945b8f1b8384/users/ea1954ef80404716929008b389c9e990/roles/e07cc5009f624eb9912a837c934e68aa` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:57.124012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce46a02781c74eb5a27d945b8f1b8384, user_id=ea1954ef80404716929008b389c9e990, role_id=e07cc5009f624eb9912a837c934e68aa)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:57.126874 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:57.136771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1449/2877] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:41:57 2022] PUT /identity/v3/projects/ce46a02781c74eb5a27d945b8f1b8384/users/ea1954ef80404716929008b389c9e990/roles/e07cc5009f624eb9912a837c934e68aa => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:41:57.147171 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e988b-ea12-41fb-b88c-3d3306d30a70 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:57.147503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e988b-ea12-41fb-b88c-3d3306d30a70 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:57.147782 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e988b-ea12-41fb-b88c-3d3306d30a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:57.174385 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-184e988b-ea12-41fb-b88c-3d3306d30a70 None None] MFA Rules not processed for user `ea1954ef80404716929008b389c9e990`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:41:57.175614 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-184e988b-ea12-41fb-b88c-3d3306d30a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:57.240672 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1429/2878] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:41:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:41:57.251141 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:41:57.253235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:41:57.309782 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b90a74-85a5-4d5d-a6e0-ef83f749d30a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:41:57.313472 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc2c6-ee20-4c7d-a221-458bd2d7dd6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:41:57.314107 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc2c6-ee20-4c7d-a221-458bd2d7dd6e service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:41:57.314391 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc2c6-ee20-4c7d-a221-458bd2d7dd6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:41:57.314669 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc2c6-ee20-4c7d-a221-458bd2d7dd6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:41:57.333811 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:41:57.371043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fc2c6-ee20-4c7d-a221-458bd2d7dd6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:41:57.372672 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fc2c6-ee20-4c7d-a221-458bd2d7dd6e service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:41:57.382004 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1450/2879] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:41:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:42:01.539959 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e988b-ea12-41fb-b88c-3d3306d30a70 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:01.541344 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-184e988b-ea12-41fb-b88c-3d3306d30a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:01.553579 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e988b-ea12-41fb-b88c-3d3306d30a70 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:01.557091 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56db94-97f7-4fc9-a08c-fafac0615276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:01.557726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56db94-97f7-4fc9-a08c-fafac0615276 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:01.558008 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56db94-97f7-4fc9-a08c-fafac0615276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:01.558283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56db94-97f7-4fc9-a08c-fafac0615276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:01.568115 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56db94-97f7-4fc9-a08c-fafac0615276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:01.569525 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56db94-97f7-4fc9-a08c-fafac0615276 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:01.579180 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1430/2880] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:42:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 17:42:06.422270 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc2c6-ee20-4c7d-a221-458bd2d7dd6e service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:06.486768 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc2c6-ee20-4c7d-a221-458bd2d7dd6e service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:06.491246 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:06.492174 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:06.492452 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:06.492724 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] PATH_INFO: `/v3/users/f84fca1d74de49f48a7f629ef9fb54a9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:06.493888 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84fca1d74de49f48a7f629ef9fb54a9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:06.495825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:06.541688 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84fca1d74de49f48a7f629ef9fb54a9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:06.542522 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84fca1d74de49f48a7f629ef9fb54a9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:06.550972 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84fca1d74de49f48a7f629ef9fb54a9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:06.563018 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84fca1d74de49f48a7f629ef9fb54a9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:06.563695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84fca1d74de49f48a7f629ef9fb54a9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:06.576212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1451/2881] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:42:06 2022] DELETE /identity/v3/users/f84fca1d74de49f48a7f629ef9fb54a9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:42:06.580757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb09cf-cd09-4509-b6b4-2d0b07488243 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:06.581271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb09cf-cd09-4509-b6b4-2d0b07488243 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:06.581389 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb09cf-cd09-4509-b6b4-2d0b07488243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:06.599412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e1bb09cf-cd09-4509-b6b4-2d0b07488243 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:42:06.600607 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e1bb09cf-cd09-4509-b6b4-2d0b07488243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:06.611454 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:06.661130 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1431/2882] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:42:06.671180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:06.672629 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:06.714729 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:06.730984 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68eed6-1949-411a-af8f-16c968234f67 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:06.735392 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f9b85-c71d-452a-a258-8a561f2ede28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:06.736047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f9b85-c71d-452a-a258-8a561f2ede28 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:06.736328 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f9b85-c71d-452a-a258-8a561f2ede28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:06.736602 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f9b85-c71d-452a-a258-8a561f2ede28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:06.745513 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f9b85-c71d-452a-a258-8a561f2ede28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:06.746945 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f9b85-c71d-452a-a258-8a561f2ede28 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:06.756771 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1452/2883] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:42:07.034365 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb09cf-cd09-4509-b6b4-2d0b07488243 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:07.035966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e1bb09cf-cd09-4509-b6b4-2d0b07488243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:07.049467 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb09cf-cd09-4509-b6b4-2d0b07488243 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:07.054452 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:07.055348 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:07.055646 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:07.055926 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] PATH_INFO: `/v3/projects/636d745a71364d3a92df90e19d438811` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:07.058406 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] RBAC: Authorizing `identity:delete_project(project_id=636d745a71364d3a92df90e19d438811)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:07.061143 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:07.107866 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636d745a71364d3a92df90e19d438811', 'request_id': 'req-b9a7e22a-140f-4903-a59c-b123a0c7d48b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.114837 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636d745a71364d3a92df90e19d438811', 'request_id': 'req-b9a7e22a-140f-4903-a59c-b123a0c7d48b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.120966 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636d745a71364d3a92df90e19d438811', 'request_id': 'req-b9a7e22a-140f-4903-a59c-b123a0c7d48b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.132546 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] The token cache is being invalidate because project 636d745a71364d3a92df90e19d438811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:42:07.132836 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.133465 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.135006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1432/2884] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:42:07 2022] DELETE /identity/v3/projects/636d745a71364d3a92df90e19d438811 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:42:07.139287 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f9b85-c71d-452a-a258-8a561f2ede28 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:07.148881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f9b85-c71d-452a-a258-8a561f2ede28 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:07.205825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:07.206719 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:07.207003 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:07.207311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] PATH_INFO: `/v3/users/ea1954ef80404716929008b389c9e990` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:07.208787 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1954ef80404716929008b389c9e990)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:07.210728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:07.256091 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1954ef80404716929008b389c9e990'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.256725 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1954ef80404716929008b389c9e990'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.263828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1954ef80404716929008b389c9e990'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.275440 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1954ef80404716929008b389c9e990'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.276070 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4fb2f592-a1f4-46a5-bddb-349f26a73ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1954ef80404716929008b389c9e990'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.289263 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1453/2885] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:42:07 2022] DELETE /identity/v3/users/ea1954ef80404716929008b389c9e990 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:42:07.557486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:07.559002 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:07.573928 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7e22a-140f-4903-a59c-b123a0c7d48b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:07.630613 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:07.631519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:07.631801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:07.632076 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] PATH_INFO: `/v3/projects/ce46a02781c74eb5a27d945b8f1b8384` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:07.633253 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce46a02781c74eb5a27d945b8f1b8384)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:07.635144 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:07.674257 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:07.681283 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce46a02781c74eb5a27d945b8f1b8384', 'request_id': 'req-a35e8f0d-a599-437f-99da-e6b80bd075d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.689556 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce46a02781c74eb5a27d945b8f1b8384', 'request_id': 'req-a35e8f0d-a599-437f-99da-e6b80bd075d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.695784 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce46a02781c74eb5a27d945b8f1b8384', 'request_id': 'req-a35e8f0d-a599-437f-99da-e6b80bd075d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.705980 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] The token cache is being invalidate because project ce46a02781c74eb5a27d945b8f1b8384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:42:07.706246 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.706874 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a35e8f0d-a599-437f-99da-e6b80bd075d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:07.708494 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1433/2886] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:42:07 2022] DELETE /identity/v3/projects/ce46a02781c74eb5a27d945b8f1b8384 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:42:07.715665 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f53a53-002e-4bea-948e-685327d77687 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:07.716755 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f53a53-002e-4bea-948e-685327d77687 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:07.717043 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f53a53-002e-4bea-948e-685327d77687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:07.735235 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-07f53a53-002e-4bea-948e-685327d77687 None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:42:07.736451 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-07f53a53-002e-4bea-948e-685327d77687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:07.795152 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1454/2887] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 17:42:07.804409 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898eb2b-0d3f-4732-986a-7a3d1befeafb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:07.804698 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898eb2b-0d3f-4732-986a-7a3d1befeafb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:07.804975 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898eb2b-0d3f-4732-986a-7a3d1befeafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:07.823930 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6898eb2b-0d3f-4732-986a-7a3d1befeafb None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:42:07.825157 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6898eb2b-0d3f-4732-986a-7a3d1befeafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:07.833021 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1434/2888] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:42:07.837236 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f53a53-002e-4bea-948e-685327d77687 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:07.842373 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:07.848636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f53a53-002e-4bea-948e-685327d77687 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:07.853813 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:07.854460 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:07.854738 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:07.855007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:07.855772 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 17:42:07.856061 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:07.858299 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:07.869664 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1455/2889] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 17:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 17:42:07.874603 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898eb2b-0d3f-4732-986a-7a3d1befeafb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:07.884120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898eb2b-0d3f-4732-986a-7a3d1befeafb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:07.888801 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f035-67fa-4e9a-9b53-9d3e835ab5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:07.889450 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f035-67fa-4e9a-9b53-9d3e835ab5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:07.889730 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f035-67fa-4e9a-9b53-9d3e835ab5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:07.890006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f035-67fa-4e9a-9b53-9d3e835ab5f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:07.890702 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56f035-67fa-4e9a-9b53-9d3e835ab5f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:07.892616 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56f035-67fa-4e9a-9b53-9d3e835ab5f1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:07.923052 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1435/2890] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 17:42:07 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:42:07.927179 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:07.938214 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d209df9-a136-42b1-817c-402da5557cfe admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:07.996224 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2a8d4-14be-4eca-a1c9-24ff2b1723bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:07.996812 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2a8d4-14be-4eca-a1c9-24ff2b1723bf admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:07.997212 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2a8d4-14be-4eca-a1c9-24ff2b1723bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:07.997639 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2a8d4-14be-4eca-a1c9-24ff2b1723bf admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:07.998430 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2a8d4-14be-4eca-a1c9-24ff2b1723bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:08.000386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2a8d4-14be-4eca-a1c9-24ff2b1723bf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:08.019918 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1456/2891] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 17:42:07 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 17:42:08.024055 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f035-67fa-4e9a-9b53-9d3e835ab5f1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:08.035326 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f035-67fa-4e9a-9b53-9d3e835ab5f1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:08.040107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d59d4a-9612-494f-a191-ea4f2d434a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:08.040794 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59d4a-9612-494f-a191-ea4f2d434a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:08.041075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59d4a-9612-494f-a191-ea4f2d434a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:08.041351 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59d4a-9612-494f-a191-ea4f2d434a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:08.042107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d59d4a-9612-494f-a191-ea4f2d434a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:08.043719 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d59d4a-9612-494f-a191-ea4f2d434a38 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:08.054994 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1436/2892] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:42:08.059024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2a8d4-14be-4eca-a1c9-24ff2b1723bf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:08.068881 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2a8d4-14be-4eca-a1c9-24ff2b1723bf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:08.073501 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360716b-ac79-4a46-afcb-07d8a75fe7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:08.074148 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360716b-ac79-4a46-afcb-07d8a75fe7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:08.074429 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360716b-ac79-4a46-afcb-07d8a75fe7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:08.074704 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360716b-ac79-4a46-afcb-07d8a75fe7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:08.075658 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360716b-ac79-4a46-afcb-07d8a75fe7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:08.077047 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360716b-ac79-4a46-afcb-07d8a75fe7f6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:08.086825 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1457/2893] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 17:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 17:42:08.091306 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d59d4a-9612-494f-a191-ea4f2d434a38 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:08.101530 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d59d4a-9612-494f-a191-ea4f2d434a38 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:08.106215 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:08.108179 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:08.108500 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:08.108810 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] PATH_INFO: `/v3/projects/79fa35399cc74aca9491b2073e051427/users/a6f219d6718741929948143f2a28adf6/roles/4ef78f9a0aba4d74b1d65fd0740cdc50` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:08.124195 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79fa35399cc74aca9491b2073e051427, user_id=a6f219d6718741929948143f2a28adf6, role_id=4ef78f9a0aba4d74b1d65fd0740cdc50)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:08.127079 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:08.141503 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1437/2894] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 17:42:08 2022] PUT /identity/v3/projects/79fa35399cc74aca9491b2073e051427/users/a6f219d6718741929948143f2a28adf6/roles/4ef78f9a0aba4d74b1d65fd0740cdc50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:42:08.152619 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd3d0-f321-4d98-9e31-b6e4468b2ea6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:08.152938 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd3d0-f321-4d98-9e31-b6e4468b2ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:08.153777 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd3d0-f321-4d98-9e31-b6e4468b2ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:08.180507 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0c4fd3d0-f321-4d98-9e31-b6e4468b2ea6 None None] MFA Rules not processed for user `a6f219d6718741929948143f2a28adf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:42:08.181708 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0c4fd3d0-f321-4d98-9e31-b6e4468b2ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:08.244301 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1458/2895] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:42:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:42:08.256120 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:08.257893 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:08.314577 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d7f1e-ed16-44f4-88a6-f08287e8dc02 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:08.318245 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f71e96-55d3-4bac-bc3f-0d1058198b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:08.318887 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f71e96-55d3-4bac-bc3f-0d1058198b3b service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:08.319170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f71e96-55d3-4bac-bc3f-0d1058198b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:08.319488 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f71e96-55d3-4bac-bc3f-0d1058198b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:08.328374 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f71e96-55d3-4bac-bc3f-0d1058198b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:08.329799 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f71e96-55d3-4bac-bc3f-0d1058198b3b service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:08.338456 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1438/2896] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:42:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:42:10.862467 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fd3d0-f321-4d98-9e31-b6e4468b2ea6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:10.922267 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:10.928536 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fd3d0-f321-4d98-9e31-b6e4468b2ea6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:10.935641 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:10.936539 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:10.936826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:10.937125 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] PATH_INFO: `/v3/users/a6f219d6718741929948143f2a28adf6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:10.938342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f219d6718741929948143f2a28adf6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:10.940326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:10.990419 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f219d6718741929948143f2a28adf6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:10.991044 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f219d6718741929948143f2a28adf6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:10.999131 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f219d6718741929948143f2a28adf6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:11.011636 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f219d6718741929948143f2a28adf6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:11.012444 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f219d6718741929948143f2a28adf6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:11.025074 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1459/2897] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:42:10 2022] DELETE /identity/v3/users/a6f219d6718741929948143f2a28adf6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:42:11.031234 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48dcc-e172-4e62-9c3f-11ad1787a78b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:11.031808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48dcc-e172-4e62-9c3f-11ad1787a78b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:11.031808 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48dcc-e172-4e62-9c3f-11ad1787a78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:11.051311 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-40b48dcc-e172-4e62-9c3f-11ad1787a78b None None] MFA Rules not processed for user `04bc439d29964fc096fdffcc884a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:42:11.052565 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-40b48dcc-e172-4e62-9c3f-11ad1787a78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:11.119540 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1439/2898] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:42:11.130349 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:11.132101 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:11.189227 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec87fc-4307-41f7-b21a-03ca90095102 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:11.193822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:11.194470 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:11.194752 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:11.195024 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:11.204600 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:11.206114 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:11.216492 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1460/2899] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:42:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:42:11.528088 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48dcc-e172-4e62-9c3f-11ad1787a78b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:11.530489 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-40b48dcc-e172-4e62-9c3f-11ad1787a78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:11.543107 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48dcc-e172-4e62-9c3f-11ad1787a78b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:11.559213 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:11.560337 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:11.560622 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:11.560948 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] PATH_INFO: `/v3/projects/79fa35399cc74aca9491b2073e051427` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:11.562756 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79fa35399cc74aca9491b2073e051427)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:11.565278 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:11.568167 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:11.629974 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79fa35399cc74aca9491b2073e051427', 'request_id': 'req-2439c6bc-2f0e-494f-9c5f-14fed12cb053'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:11.638014 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79fa35399cc74aca9491b2073e051427', 'request_id': 'req-2439c6bc-2f0e-494f-9c5f-14fed12cb053'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:11.646053 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79fa35399cc74aca9491b2073e051427', 'request_id': 'req-2439c6bc-2f0e-494f-9c5f-14fed12cb053'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:11.658814 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] The token cache is being invalidate because project 79fa35399cc74aca9491b2073e051427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:42:11.659098 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:11.659776 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2439c6bc-2f0e-494f-9c5f-14fed12cb053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:42:11.662082 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1440/2900] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:42:11 2022] DELETE /identity/v3/projects/79fa35399cc74aca9491b2073e051427 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:42:21.211543 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:21.655729 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:47.398593 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:47.400009 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:47.456889 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff8ae-78d5-4a1e-935f-06418c6211d4 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:47.505051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57af250e151241f4afc9b38d51a5fbdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:47.505695 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:47.505980 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:47.506254 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:47.508059 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:47.565760 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:47.567180 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:47.575937 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1461/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:42:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:42:57.524486 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb029-b86d-4885-940f-da13e48b6330 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:57.525100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb029-b86d-4885-940f-da13e48b6330 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:57.525100 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb029-b86d-4885-940f-da13e48b6330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:57.543531 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-52cdb029-b86d-4885-940f-da13e48b6330 None None] MFA Rules not processed for user `4a827bedc4b84508bb7770a09efee3f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 17:42:57.544735 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-52cdb029-b86d-4885-940f-da13e48b6330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:57.552522 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1441/2902] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 17:42:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 17:42:57.571817 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=68 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:42:58.311625 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:42:58.313162 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:58.383481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef0d32-1100-4eda-8f8f-f301bca8e368 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:42:58.388372 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:42:58.389008 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:42:58.389286 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:42:58.389652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:42:58.391433 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:42:58.402386 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:42:58.404017 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:42:58.414370 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1462/2903] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:42:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:43:00.844885 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdb029-b86d-4885-940f-da13e48b6330 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:43:00.846882 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-52cdb029-b86d-4885-940f-da13e48b6330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:43:00.936373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdb029-b86d-4885-940f-da13e48b6330 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:43:00.942688 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:43:00.943373 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:43:00.943664 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:43:00.943944 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:43:00.945760 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:43:00.959317 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:43:00.960973 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:43:00.973619 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1442/2904] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:43:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:43:08.410223 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=32 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:43:10.969056 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=37 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:43:11.351020 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:43:11.365281 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603bb1f-2062-40f9-9160-f524430cc1b3 service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:43:11.371728 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:43:11.372381 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:43:11.372654 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:43:11.372931 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:43:11.375279 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:43:11.388914 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:43:11.391239 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:43:11.402132 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1463/2905] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:43:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:43:13.785564 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:43:13.787032 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:43:13.842807 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6cc60-58d5-4726-b131-2f824554bd28 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:43:13.846495 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:43:13.847138 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:43:13.847448 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:43:13.847726 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:43:13.849501 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:43:13.911727 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:43:13.913204 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:43:13.922827 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1443/2906] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:43:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:43:21.397482 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=6 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:43:23.918397 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=50 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:44:50.702797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:44:50.704216 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:44:50.718315 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb93bdd-9ee5-4ef2-b1a6-755283ccb5e5 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:44:50.723828 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:44:50.724476 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:44:50.724778 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:44:50.725051 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:44:50.726824 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:44:50.738494 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:44:50.740872 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:44:50.750674 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1464/2907] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:44:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:45:00.746382 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=6 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:45:05.580818 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:45:05.594164 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c781e9-74f4-4223-9221-35b01be993bb service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:45:05.601287 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9888d2731f4117b5fc2c3a720931c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:45:05.602040 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:45:05.602393 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:45:05.602733 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:45:05.604670 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:45:05.618390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:45:05.620075 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:45:05.632238 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1444/2908] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:45:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:45:15.626734 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:45:17.748409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:45:17.758894 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d10ba-c38d-4289-8d69-bf7d5d69cd01 service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:45:17.762409 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71a48b3b4fa94798b158488f08bae743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:45:17.763034 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:45:17.763311 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:45:17.763611 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:45:17.765342 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:45:17.829652 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:45:17.831096 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:45:17.840797 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1465/2909] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:45:20.816878 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:45:20.829433 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1790f9-7cd6-4010-a01a-b220e3fb42a7 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:45:20.836170 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:45:20.837060 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:45:20.837440 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:45:20.837787 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:45:20.839714 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:45:20.854196 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:45:20.855963 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:45:20.868832 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1445/2910] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:45:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:45:24.005264 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:45:24.007007 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:45:24.020322 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50827c15-3cdc-44ee-bb13-bdb2291591af service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:45:24.025925 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:45:24.026678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:45:24.026961 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:45:24.027249 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] PATH_INFO: `/v3/users/4a827bedc4b84508bb7770a09efee3f5` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:45:24.028500 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a827bedc4b84508bb7770a09efee3f5)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:45:24.030546 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:45:24.084821 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a827bedc4b84508bb7770a09efee3f5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.085667 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a827bedc4b84508bb7770a09efee3f5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.092804 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a827bedc4b84508bb7770a09efee3f5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.104488 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a827bedc4b84508bb7770a09efee3f5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.105145 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a827bedc4b84508bb7770a09efee3f5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.118086 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1466/2911] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:45:24 2022] DELETE /identity/v3/users/4a827bedc4b84508bb7770a09efee3f5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:45:24.291081 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:45:24.303549 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48afc3a-709b-4457-923d-d0dc674e0333 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:45:24.360955 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:45:24.361824 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:45:24.362111 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:45:24.362390 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] PATH_INFO: `/v3/projects/3785f78e6d1048cfa0b5b123fdd909c8` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:45:24.363595 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3785f78e6d1048cfa0b5b123fdd909c8)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:45:24.365519 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:45:24.413066 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3785f78e6d1048cfa0b5b123fdd909c8', 'request_id': 'req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.421128 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3785f78e6d1048cfa0b5b123fdd909c8', 'request_id': 'req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.429243 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3785f78e6d1048cfa0b5b123fdd909c8', 'request_id': 'req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.439865 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] The token cache is being invalidate because project 3785f78e6d1048cfa0b5b123fdd909c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:45:24.440151 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.441068 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.442636 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1446/2912] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:45:24 2022] DELETE /identity/v3/projects/3785f78e6d1048cfa0b5b123fdd909c8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:45:24.447592 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:45:24.448960 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:45:24.461836 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252d44c-5e67-476d-9659-99e6cd50dde6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:45:24.517407 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:45:24.518282 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:45:24.518558 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:45:24.518826 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] PATH_INFO: `/v3/users/9f9a2922508546ac9fbbcf643e5340c0` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:45:24.520021 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f9a2922508546ac9fbbcf643e5340c0)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:45:24.522079 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:45:24.527449 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:45:24.566207 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f9a2922508546ac9fbbcf643e5340c0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.566844 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f9a2922508546ac9fbbcf643e5340c0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.574363 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f9a2922508546ac9fbbcf643e5340c0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.585939 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f9a2922508546ac9fbbcf643e5340c0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.586562 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f9a2922508546ac9fbbcf643e5340c0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.598012 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1467/2913] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 17:45:24 2022] DELETE /identity/v3/users/9f9a2922508546ac9fbbcf643e5340c0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:45:24.771047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:45:24.773591 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:45:24.786271 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622c080-09e2-4491-a4e2-9ea31bf6a9fd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:45:24.845417 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04bc439d29964fc096fdffcc884a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b7d62044f5549a6b53772308a090b05', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:45:24.846281 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:45:24.846563 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:45:24.846841 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] PATH_INFO: `/v3/projects/a52ea793d7ce46f59ed1392b1b987de5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:45:24.848047 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a52ea793d7ce46f59ed1392b1b987de5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:45:24.850262 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:45:24.871013 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:45:24.898654 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a52ea793d7ce46f59ed1392b1b987de5', 'request_id': 'req-dd95bbc7-6957-4057-a764-fdd03899b07c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.905006 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a52ea793d7ce46f59ed1392b1b987de5', 'request_id': 'req-dd95bbc7-6957-4057-a764-fdd03899b07c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.911295 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a52ea793d7ce46f59ed1392b1b987de5', 'request_id': 'req-dd95bbc7-6957-4057-a764-fdd03899b07c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.921849 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] The token cache is being invalidate because project a52ea793d7ce46f59ed1392b1b987de5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 17:45:24.922133 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.922757 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-dd95bbc7-6957-4057-a764-fdd03899b07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 17:45:24.924412 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1447/2914] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 17:45:24 2022] DELETE /identity/v3/projects/a52ea793d7ce46f59ed1392b1b987de5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 17:45:34.594621 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:45:34.918880 dv-cinder-843729-1-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 17:49:56.675942 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 17:49:56.677333 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:49:56.816481 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4c0de-5cee-476f-a392-4a3d0613f303 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 17:49:56.873678 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291951d5-212c-4d2a-8fc1-b4532a023a70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d2bfa61bb7245579090eb060baf69ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceee56bba3764f4ca3ec96f17d12848b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 17:49:56.874326 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291951d5-212c-4d2a-8fc1-b4532a023a70 service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 17:49:56.874607 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291951d5-212c-4d2a-8fc1-b4532a023a70 service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 17:49:56.874879 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291951d5-212c-4d2a-8fc1-b4532a023a70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 17:49:56.876655 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-291951d5-212c-4d2a-8fc1-b4532a023a70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 17:49:56.887822 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291951d5-212c-4d2a-8fc1-b4532a023a70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 17:49:56.889400 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291951d5-212c-4d2a-8fc1-b4532a023a70 service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 17:49:56.899963 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1468/2915] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 17:49:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 17:50:06.895694 dv-cinder-843729-1-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}